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The  results  of  the  study  determined  that  the  FO  training  device  concept 
is  valid.  All  the  tested  training  programs  employing  the  BT33  resulted  in 
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training  program. 
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FOREWORD 


The  research  reported  here  was  performed  by  the  Army  Research 
Institute's  Fort  Benning  Field  Unit.  It  is  part  of  an  ongoing  program 
of  research  directed  toward  development  of  cost  effective  methods  for 
individual  and  collective  training.  This  program  includes  research  on 
multiple  aspects  of  the  design,  development,  evaluation,  and  integra- 
tion of  cost  and  training  effective  training  systems  for  the  U.S.  Army. 

This  report  presents  an  analysis  of  the  relative  training  effec- 
tiveness of  the  BT33,  a Swedish-developed  Forward  Observer  (FO)  train- 
ing device.  The  analysis  was  designed  in  response  to  a request  by  the 
USA  Field  Artillery  School  (USAFAS)  and  seconded  by  USA  Training  Support 
Center  Training  Devices  Directorate  (ATSC  TDD) . The  study  was  to  vali- 
date the  concept  that  FO  training  devices  could  yield  effective  training 
with  potential  cost  savings.  Results  of  the  study  were  to  be  used  as 
input  to  the  cost  and  training  effectiveness  analysis  (CTEA)  being  con- 
ducted on  the  Observed  Fire  Trainer  under  development  by  the  Army. 

The  research  was  executed  at  USAFAS  using  the  FA  Basic  Officer 
Course  as  the  vehicle.  Close  coordination  between  ARI  and  USAFAS,  to- 
gether with  dedicated  test  supervision  by  the  USAFAS  Study  Director, 

LTC  Lyle  Butler,  Jr.,  and  the  Project  Officer,  CPT  Spencer  A.  Fisher, 
resulted  in  a successful  test  of  BT33  training  effectiveness. 

ARI  research  in  training  systems  development  is  conducted  as  an 
inhouse  effort  augmented  by  contracts  with  organizations  selected  as 
having  unique  capabilities  for  research  in  the  area.  This  study  was 
performed  by  ARI  personnel  from  the  Fort  Benning  Field  Unit.  The  proj- 
ect was  conducted  as  part  of  Army  Project  2Q763731A773 , FY  76  Work 
Program,  and  Army  Project  2Q763743A773,  FY  77.  It  was  directly  re- 
sponsive to  the  requireifients  of  the  USAFAS  and  ATSC  TDD. 
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BT33  TRAINING  EFFECTIVENESS  ANALYSIS 


BRIEF 


Requirement : 


To  validate  the  concept  of  Forward  Observer  (FO)  training  device 
cost  and  training  benefits.  This  is  in  support  of  a program  to  develop 
a unit  FO  training  device  for  the  U.S.  Army. 


Procedure : 

A Swedish  institutional  FO  training  device,  the  BT33,  was  obtained 
for  testing  at  the  U.S.  Army  Field  Artillery  School  (USAFAS) . It  is 
similar  in  overall  simulation  capability  to  the  Observed  Fire  Trainer 
(OFT)  being  developed  in  the  United  States  for  a more  rugged  unit  train- 
ing environment.  FO  performance  resulting  from  the  current  FO  train- 
ing course  (which  employs  costly  live  fire  exercises)  was  compared  to 
FO  performances  resulting  from  seven  variations  of  the  course  using 
the  BT33. 


Findings : 


The  findings  indicate  that  FO  training  devices  like  the  BT33  can 
be  effective  additions  to  FO  training  programs.  Specifically: 

• The  FO  training  device  concept  is  valid.  All  the  tested 
training  programs  employing  the  BT33  resulted  in  training 
that  appeared  overall  to  be  as  good  as  or  better  than  the 
current  training  program. 

• Most  of  the  problems  noted  in  the  BT33  (e.g.,  poor  target 
displays,  poor  maintainability  design)  are  being  dealt  with 
in  the  OFT  development. 

• The  most  serious  apparent  limitation  of  the  BT33,  the  two- 
dimensional  display,  as  opposed  to  a three-dimensional  dis- 
play, is  not  a problem  as  long  as  it  is  realized  that  some 
time  must  still  be  allotted  for  additional  visual  skills 
development  in  the  field  environment. 


Utilization  of  Findings: 

The  results  of  this  BT33  training  effectiveness  analysis  (TEA) 
are  being  integrated  into  the  OFT  cost  and  training  effectiveness 
analysis  (CTEA)  and  OT  II  test  plans.  These  results  are  also  being 
used  by  U SAFAS,  in  conjunction  with  other  data,  to  evaluate  the  pos 
sibility  of  retaining  the  BT33  at  USAFAS  for  institutional  training 
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BT33  TRAINING  EFFECTIVENESS  ANALYSIS 


Given  the  problems  of  increased  ammunition  costs  and  reduced 
training  manpower  and  budget  and  the  need  for  improved  effectiveness, 
the  USAFAS  would  like  to  develop  these  options:  (a)  achieve  the 
present  level  of  Forward  Observer  (FO)  capability  with  much  less  nan- 
power  and  budget;  (b)  achieve  higher  levels  of  FO  capability  within 
current  manpower  and  budget  limits;  and  (c)  achieve  some  improvements 
in  FO  capability  with  somewhat  less  manpower  and  budget.  Innovative 
training  techniques  which  hold  promise  of  providing  these  options  are 
two  very  similar  training  devices:  the  Swedish  SAAB-Scania  Fire  Con- 
trol Simulator  BT33  and  the  U.S. -developed  Observed  Fire  Trainer  (OFT). 
The  BT33  is  built  for  the  institutional  training  environment  and  is 
currently  available  for  analysis  and  evaluation.  The  OFT  is  now  under 
development  and  will  be  suitable  for  both  institutional  and  the  more 
rugged  unit  environments.  The  question  is,  can  these  devices  be  in- 
tegrated into  unit  and  institutional  FO  training  programs  so  as  to 
provide  the  above  options?  This  study  is  a part  of  the  OFT  Cost  and 
Training  Effectiveness  Analysis  (CTEA)  program  being  conducted  to 
answer  this  question. 


OBJECTIVES 

This  Training  Effectiveness  Analysis  (TEA)  of  the  BT33  has  been 
conducted  to: 

• Validate  the  concept  of  FO  training  device  utility. 

• Determine  how  such  a device  can  be  employed  in  the  institu- 
tional environment  so  as  to  realize  cost  and  training 
benefits. 

• Provide  data  needed  by  the  OFT  CTEA  study. 

• Provide  information  useful  to  finalization  of  the  OFT  design. 


METHOD 

Quantitative  data  were  obtained  on  the  criterion  performance  of 
students  trained  under  each  of  seven  different  experimental  training 
programs  employing  the  BT33.  These  data  were  compared  to  data  on  the 
performance  of  students  trained  under  the  current  training  program. 

The  seven  experimental  training  programs  varied  in  the  extent  to  which 
the  BT33  was  used  in  place  of  live  fire  and  the  particular  event (s) 
for  which  it  was  substituted.  The  collection  and  comparison  of  these 
data  provided  a quantitative  basis  for  evaluating  the  relative  training 
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effectiveness  of  the  BT33,  used  in  alternative  ways,  as  compared  to 
the  current  training  program. 

Questionnaire  data  were  also  obtained  from  students,  instructors, 
and  BT33  operators.  These  data  provided  qualitative  information  re- 
garding good  and  poor  aspects  of  the  BT33  and  the  acceptability  of  it 
to  students  and  instructors  for  training  and  evaluation  purposes. 


SUMMARY  CONCLUSIONS  AND  RECOMMENDATIONS 

The  results  of  this  BT33  TEA  study  establish  the  following: 

e The  FO  training  device  concept  is  valid.  More  specifically, 
all  the  tested  training  programs  employing  the  BT33  resulted 
in  training  that  appeared  overall  to  be  as  good  as  or  better 
than  the  current  training  program. 

e Most  of  the  problems  noted  in  the  BT33  (e.g.,  poor  target 
displays,  poor  maintainability  design)  are  being  dealt  with 
in  the  OFT  development. 

e The  most  serious  apparent  limitation  of  the  BT33,  the  two- 
dimensional  display,  as  opposed  to  a three-dimensional  dis- 
play, is  not  a problem  as  long  as  it  is  realized  that  some 
time  must  be  allotted  for  additional  visual  skills  develop- 
ment in  the  field  environment. 

These  conclusions  indicate  that  FO  training  devices  (like  the  BT33) 
cam  be  effective  additions  to  FO  training  programs . The  following 
recommendations  are  made: 

• USAFAS  should  study  the  detailed  results  carefully  for  impli- 
cations for  maximizing  the  training  effectiveness  of  such  de- 
vices in  training  programs . 

• When  the  BT33/OFT  is  used  to  a maximum  extent,  with  simula- 
tion of  several  live-fire  events,  performance  on  the  first 
actual  live-fire  exercise  should  be  judged  as  a transitional 
shoot  and  evaluated  accordingly. 


BACKGROUND 

General 

The  Forward  Observer  (FO)  is  responsible  for  acquiring  targets  and 
directing  artillery  fire  to  suppress  or  destroy  the  target.  The  basic 
skills  training  of  Artillery  FOs  is  accomplished  by  both  classroom  in- 
struction and  live  fire  exercises.  Currently,  the  major  training  devices 
(simulators)  used  in  the  U.S.  Army  Field  Artillery  School  (USAFAS)  13- 
week  Gunnery  training  programs  are  the  "Puffboard"  and  the  M-31  14.5mm 
subcaliber  trainer.  The  "Puffboard"  is  a device  which  allows  a puff  of 
smoke  to  burst  from  a terrain  mockup  in  response  to  a simulated  round. 

The  M-31  is  a subcaliber  device  that  allows  FO  training  on  1/10  scale 
ranges.  All  other  FO  skills  training  takes  place  in  the  field  and  re- 
quires expensive  ammunition  and  firing  battery  support. 

Given  the  problems  of  reduced  training  manpower  and  budget,  on  the 
one  hand,  and  the  need  for  improved  effectiveness,  on  the  other  hand, 
the  USAFAS  would  like  to  develop  these  options:  (a)  achieve  the  present 
level  of  FO  capability  with  much  less  manpower  and  budget,  (b)  achieve 
higher  levels  of  FO  capability  within  current  manpower  and  budget  limits, 
and  (c)  achieve  some  improvements  in  FO  capability  with  somewhat  less 
manpower  and  budget.  Innovative  training  techniques  which  hold  promise 
of  providing  these  options  are  two  very  similar  training  devices:  the 
Swedish  SAAB-Scania  Fire  Control  Simulator  BT33  and  the  U.S. -developed 
Observed  Fire  Trainer  (OFT) . The  BT33  is  built  for  institutional  train- 
ing environments  (like  USAFAS)  and  is  currently  available  for  analysis 
and  evaluation.  The  OFT  is  now  under  development  and  will  be  suitable 
for  both  institutional  and  unit  environments.  The  question  is,  can 
these  devices  be  integrated  into  unit  and  institutional  FO  training  pro- 
grams so  as  to  provide  the  above  options?  This  study  is  a part  of  the 
OFT  CTEA  program  being  conducted  to  answer  this  question. 


This  BT33  TEA  was  designed  to  (a)  validate  the  utility  of  the  FO 
training  device  concept;  (b)  assess  the  relative  effectiveness  of  the 
alternative  ways  the  BT33  can  be  used  in  FO  training  at  USAFAS;  (c)  pro- 
vide preliminary  data  for  input  to  the  CTEA  for  the  OFT;  and  (d)  provide 
design  and  performance  data  on  the  BT33  device  for  consideration  in  de- 
velopment of  the  OFT.  The  study  objectives  and  methodology  developed 
to  satisfy  these  purposes  are  described  in  the  next  major  section. 
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System  Descriptions  and  Potential  Benefits 


The  BT33  fire  control  simulator  consists  of  the  following  main 
parts:  target  area  screen,  central  unit,  display  and  burst  projection 
units,  and  target  display  unit.  The  BT33  simulator  is  designed  to  be 
sited  indoors.  The  battlefield  panorama,  including  the  target  area, 
is  projected  on  a screen  by  means  of  a slide  projector.  The  observers 
can  see  the  target  area  from  their  places  in  the  lecture  room  as  from 
observation  posts.  The  instructor  indicates  the  target  or  activities 
in  the  target  area  by  means  of  the  target  display  unit.  The  students 
observe  the  zone  before  them  and  use  normal  RTO  communication  procedures 
to  make  input  to  the  device  operator  who  simulates  the  Fire  Direction 
Center  (FDC) . The  device  operator  feeds  data  into  the  control  panel  of 
the  central  unit  as  necessary  to  implement  student  calls  for  fire.  The 
positions  of  the  bursts  in  the  target  area  are  calculated  by  the  central 
unit.  Burst  positions  simulated  by  symbols  are  projected  in  the  target 
area  by  the  burst  projectors.  The  size  and  type  of  symbols  are  deter- 
mined by  the  central  unit  to  correspond  to  the  specific  call  for  fire 
of  the  student. 

The  OFT  is  quite  similar  in  overall  configuration  and  operation  to 
the  BT33.  The  main  configuration  difference  (other  than  unitary  con- 
struction and  "ruggedizing"  of  the  design)  is  that  the  students  and  the 
projection  equipment  are  closer  to  the  OFT  target  area  display  screen 
and,  consequently,  fewer  students  can  be  accommodated  at  one  time.  The 
principal  operation  difference  is  that  certain  OFT  fire  control  func- 
tions have  been  automated,  enabling  the  OFT  instructor  to  perform  both 
instructional  and  device  operation  roles  (in  the  BT33,  two  persons  are 
required  to  perform  these  roles) . The  principal  display  difference  is 
that  the  OFT  provides  more  realistic  burst  and  target  representations. 

Potential  training  device  benefits  include  the  following: 

• A better  overall  training  effect  by  variations  in  artillery 
firing:  impact  burst,  air  burst,  illumination  missions,  smoke, 
and  firing  against  advancing  tanks  and  ships.  Variations  are 
possible  with  respect  to  burst  duration,  angle  of  impact,  fir- 
ing direction,  terrain  scenes,  dispersion,  target  movement  and 
type,  and  number  of  firing  guns.  Experience  gained  from  live 
firing  at  USAFAS  is  limited  in  number  of  guns  used  (often  one 
platoon,  two  guns),  the  resemblance  of  terrain  as  compared  to 
actual  combat  arenas,  and  the  exclusive  use  of  nonmoving 
targets . 

• A shorter  training  time  to  achieve  a fixed  performance  level. 

• A work  environment  that  is  stimulating. 

• Ability  to  continue  training  during  bad  weather. 
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Economic  gains  by  saving  artillery  ammunition,  having  fewer 
artillery  and  FDC  support  requirements,  and  having  reduced 
instructor  requirements  for  conducting  live  fire  training. 


• Added  capability  for  training  in  night  observation  and  fire 
adjustment. 


This  study  requires  diagnostic  as  well  as  status  information.  In 
other  words,  the  need  is  not  only  to  determine  if  the  BT33  is  training 
effective  but  also  to  identify  the  conditions  under  which  it  is  most 
effective  and  the  factors  determining  effectiveness  variation.  This 
additional  information  cam  be  used  to  maximize  OFT  effectiveness  through 
inputs  to  system  design  amd  to  training  system  procedures.  Given  this 
need  for  detailed  information  about  both  causes  amd  effects,  it  was 
necessaury  to  (a)  obtain  both  student  performamce  and  subjective  opinion 
data  on  a comprehensive  measure  set  amd  (b)  obtain  criterion  data  on 
student  performances  resulting  from  alternative  BT33/live  fire  mixes. 

The  following  two  sections  discuss  the  test  design  and  analysis  pro- 
cedures used  in  the  study. 


Design  Considerations.  The  study  design  resulted  from  three  con- 
siderations: (a)  what  are  the  alternative  ways  in  which  the  BT33  could 

be  integrated  into  the  FO  training  program;  (b)  what  constitutes  the 
criterion  conditions  needed  to  demonstrate  training  effectiveness;  and 
(c)  how  cam  the  constitution  of  the  various  experimental  groups  be  con- 
trolled with  respect  to  aptitude  levels?  With  regard  to  the  first 
question,  the  BT33  and,  hence,  the  OFT  could  be  utilized  in  FO  training 
at  USAFAS  for  the  following  groups:  Field  Artillery  Officer  Basic 
Course  (FAOBC)  students.  Field  Artillery  Officer  Advamced  Course  (FAOAC) 
students.  Marine  Scout  Officer  Course  (MASOC)  students.  Advanced  Indi- 
vidual Training  (AIT)  students,  and  III  Corps  Artillery  personnel.  The 
FAOBC  program  provides  the  largest  amd  most  controllable  population  of 
beginning  students  amd  will  yield  the  highest  payoff  if  the  training 
devices  are  proven  useful.  Therefore,  the  FAOBC  program  was  selected 
as  the  test  bed  for  the  BT33  training  effectiveness  evaluation.  The 
USAFAS  Gunnery  Department  evaluated  the  FO  portion  of  their  FAOBC 
training  program  in  terms  of  apparent  BT33  capabilities  amd  identified 
three  points  in  the  prograun  at  which  the  BT33  might  well  be  substituted 
for  current  training  procedures.  These  training  events  are 


The  Puffboard  Exercise — during  Gunnery  Week  2 the  Puf fboaurd 
is  used  to  introduce  the  students  to  FO  procedures  amd  to  pro' 
vide  the  first  opportunity  for  a few  members  of  the  class  to 
practice.  This  device  was  described  earlier. 


• Daly  Hill — a live-fire  adjustment  exercise  taking  place  on 
Daly  Hill,  Fort  Sill,  during  Gunnery  week  5.  All  students 
perform  at  least  one  mission. 

• Arbuckle  Hill — a live-fire  exercise  taking  place  on  Ar buckle 
Hill,  Fort  Sill,  during  Gunnery  Week  7.  All  students  perform 
at  least  one  mission. 

Substitution  of  the  BT33  for  one  or  more  of  these  events,  plus  the  pro 
gram  as  it  is  currently  run,  constitutes  eight  experimental  conditions 
for  the  BT33  effectiveness  evaluation.  Experimental  conditions  1 
through  8 were  as  follows: 
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With  regard  to  the  second  question,  the  criterion  events  are  the 
three  live  fire  graded  exercises  which  follow  the  Arbuckle  Hill  training 
event. 

• Mobile  Shoot  One — takes  place  during  the  7th  to  9th  weeks  of 
training  and  requires  the  students  to  walk  or  ride  from  shoot 
location  to  shoot  location,  using  multiple  target  areas.  All 
students  perform  at  least  one  mission. 

• Bunker  Shoot — takes  place  during  the  8th  to  10th  weeks  of  train- 
ing and  requires  the  students  to  work  from  a bunker  and  uses  one 
target  area.  All  students  perform  at  least  one  mission. 

• Mobile  Shoot  Two — takes  place  during  the  9th  to  11th  weeks  of 
training  and,  again,  requires  the  students  to  walk  or  ride  from 
shoot  location  to  shoot  location,  using  multiple  target  areas. 
All  students  perform  at  least  one  mission. 

Performance  data  on  these  graded  events,  if  comparable,  would  permit 
evaluation  of  the  relative  effectiveness  of  the  alternative  BT33  utili- 
zation plans. 


A question  that  required  examination  was  whether  or  not  the  data 
would  be  comparable.  That  is,  would  the  sample  of  experiences  of  one 
experimental  group  on,  for  example.  Mobile  Shoot  One,  be  reasonably 
similar  to  the  sample  of  experiences  encountered  by  other  experimental 
groups,  so  that  performance  differences  between  the  groups  would  be 
primarily  a function  of  experimental  condition  differences.  There 
were  two  factors  affecting  the  comparability  of  the  data.  One  factor 
was  that  no  two  students  shoot  at  exactly  the  same  target/terrain  com- 
bination or  from  the  same  perspective.  The  other  factor  was  that  the 
mobile  shoots  take  place  alternately  on  two  ranges  which  differ  greatly 
in  the  difficulty  of  range  estimation.  With  regard  to  the  first  factor, 
it  was  determined  through  discussions  with  the  USAFAS  Gunnery  Depart- 
ment and  a tour  of  range  locations  that,  although  the  variety  of  per- 
ceptual problems  that  could  be  encountered  was  considerable,  the  sample 
of  problems  faced  by  any  one  experimental  group  would  be  reasonably 
equivalent  to  that  faced  by  other  groups.  Further,  the  assignment  of 
terrain/target  situations  to  individuals  could  be  considered  to  be  es- 
sentially random.  With  regard  to  the  second  factor,  the  difference 
between  the  two  ranges,  it  was  determined  that  estimates  of  performance 
difference  due  to  range  difference  could  be  computed  and,  hence,  sta- 
tistically accounted  for. 


With  regard  to  the  third  question,  control  of  group  composition 
with  respect  to  aptitude,  USAFAS  currently  uses  the  STEP  score*  for 
this  purpose.  Their  objective  is  to  match  student  capabilities  and 
requirements  to  instructional  techniques.  They  do  this  by  separating 
each  class  into  four  groups  based  on  STEP  scores — high,  medium  high, 
medium  low,  and  low — which  then  receive  separate  instruction.  For  the 
BT33  effectiveness  study,  however,  it  was  desired  to  compose  groups  of 
students  with  similar  capability  for  each  of  the  experimental  condi- 
tions. The  problem  was  to  develop  a way  of  classifying  students  which 
would  both  provide  the  instructors  with  easily  managed  groups  of  stu- 
dents in  terms  of  instructional  needs  and  still  satisfy  the  require- 
ments of  the  study.  After  much  discussion  a compromise  was  effected. 
Class  groups  were  assigned  to  experimental  conditions  so  that  an  equiva- 
lent sample  from  each  of  the  four  STEP-score  categories  was  obtained  for 
each  experimental  condition  over  all  classes.  Each  instructor  had  rep- 
resentatives from  only  two  of  the  categories  (high-low  or  medium  high- 
medium  low)  in  each  group.  As  a safety  precaution,  STEP  scores  were 
obtained  for  all  students  so  that  a covariance  analysis  could  be  con- 
ducted if  necessary. 


The  STEP,  or  Sequential  Tests  of  Educational  Progress,  score  provides 
an  overall  index  of  spatial,  mathematical,  reading,  and  reasoning 
skills.  The  concern  for  group  equivalence  on  this  measure  resulted 
from  the  potential  for  interaction  between  aptitude  and  training 
programs . 
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The  study  was  designed  in  two  parts:  a pilot  phase,  using  two 
classes,  in  which  the  training,  control,  and  data  collection  methods 
were  checked  out;  and  a test  phase,  using  seven  FAOBC  classes,  during 
which  the  principal  data  reported  here  were  collected.  Test  classes, 
excluding  the  pilot  phase,  were  assigned  to  the  experimental  condi- 
tions as  follows: 
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Data  Collection.  Three  types  of  data  were  collected:  performance 
data,  questionnaire  data,  and  situation  descriptive  data.  USAFAS  in- 
structors collected  all  of  the  data.  The  performance  data  were  collected 
on  students  as  they  performed  their  missions  during  the  Daly  and  Arbuckle 
Hill  training  events  and  during  each  of  the  three  criterion  events.  The 
performance  data  collected  during  the  training  events  were  used  to  assess 
learning  behaviors  which  might  explain  performance  differences  in  the 
criterion  events.  The  performance  data  collected  during  the  criterion 
events  were  used  to  assess  the  relative  training  effectiveness  of  each 
experimental  training  condition.  The  performance  data  collected  were 

1.  Accuracy  of  initial  target  location. 

2.  Accuracy  of  final  fire-for-effect  rounds. 

3.  Time  to  locate  target. 

4.  Total  mission  time. 

5.  Number  of  rounds  fired. 

6.  Procedural  errors  cut  score. 

7.  Total  cuts  score. 

Data  collected  were  separated  into  four  subgroups,  defined  by  mis- 
sion type  and  location  method:  adjust  fire  mission-grid  coordinates 
method;  adjust  fire  mission-polar  plot  method;  immediate  suppression 
mission-grid  coordinates  method;  and  immediate  suppression  mission-polar 
plot  method.  This  separation  was  necessary  due  to  performance  differ- 
ences between  these  subgroups  as  a function  of  mission  criteria  and  pro- 
cedural differences. 


The  above  performance  measures  1 through  7 assess  the  performance 
of  both  the  individual  student  and  the  system,  and  of  both  performance 
during  a mission  phase  and  in  the  overall  mission.  Items  1 and  3 above 
are  the  purest  measures  of  student  performance  per  se.  They  assess 
those  individual  skills  applied  during  the  first  phase  of  the  indirect 
fire  control  mission.  Items  2,  4,  and  5 are  measures  of  the  total  mis- 
sion performance  effectiveness  by  the  total  indirect  fire  control  sys- 
tem. These  measures  partially  reflect  the  effectiveness  of  the  FO. 

But,  they  also  include  the  effects  of  variations  among  the  FDC  and  fir- 
ing battery  components  of  the  system,  as  well  as  other  conditions — e.g., 
weather.  Variation  in  these  factors  can  be  assumed  to  be  random  varia- 
tions , not  systematically  affected  by  the  training  conditions.  To  the 
extent  that  this  is  true,  these  three  items  are  the  most  important 
measures  from  an  operational  point  of  view.  The  major  problem  in  using 
these  measures  is  that  the  random  variation  of  these  factors  may  operate 
to  conceal  differences  in  FO  effectiveness  actually  existing  as  an  ef- 
fect of  the  training  conditions.  This  is  a statistical  and  empirical 
question  to  be  answered  through  analysis.  Items  6 and  7 above  are  in- 
structor judgments  of  procedural  errors  (primarily  as  observed  in  er- 
roneous communications  to  the  FDC)  and  of  wasted  time  or  rounds  of 
ammunition.  They  are  direct  FO  performance  measures  which  assess  pro- 
cedural skills.  Data  on  these  two  measures  were  expected  to  vary  as  a 
function  of  instructor  differences  as  well  as  student  skill  differences. 
Multiple  instructors  were  randomly  assigned  to  each  condition  to  mini- 
mize the  effect  of  instructor  differences. 

Questionnaire  data  were  collected  from  both  students  and  instructors 
in  FAOBC  and  FAOAC.  Students  were  asked  to  judge  the  effectiveness  of 
the  BT33  in  training  different  aspects  of  FO  performance  (e.g.,  communi- 
cation procedures  vs.  range  estimation) , how  well  it  was  being  utilized, 
and  what  might  be  done  to  improve  its  effectiveness.  Instructors  judged 
apparent  device  effectiveness,  how  well  it  facilitated  the  instructional 
process,  and  any  problems  and  problem  solutions.  Opinions  were  also  ob- 
tained from  the  device  operators  concerning  operability,  maintainability, 
operator  training  requirements,  and  design  improvements.  The  question- 
naires are  presented  in  appendix  A. 

Situation  descriptive  data  were  collected  to  allow  an  assessment 
of  whether  or  not  bias  was  introduced  from  uncontrolled  factors  and  to 
enable  statistical  control  where  necessary.  The  principal  data  collected 
were  instructor  names,  event  dates,  event  weather  conditions,  and  in- 
structor backgrounds  and  experience. 


Analysis  Procedures 

The  Questions.  The  purposes  of  the  study  can  be  translated  into 
questions  concerning  training  effectiveness,  learning  behaviors,  device 
design,  and  device  usage: 

• What  is  the  demonstrated  relative  training  effectiveness  of  the 
BT33  in  training  for  live-fire  FO  missions? 
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• What  is  the  judged  relative  training  effectiveness  of  the 
BT33  in  training  for  live-fire  FO  missions? 

• Are  the  training  programs  differentially  effective  for  stu- 
dents of  different  aptitude  levels? 

• How  well  is  the  BT33  designed  from  a human  factors  standpoint? 

• How  well  does  the  BT33  facilitate  instruction  as  compared  to 
training  events  consisting  of  live-fire? 

• To  what  extent  do  students  and  instructors  want  to  use  the 
BT33  for  training  purposes? 

• Can  the  BT33  be  useful  in  student  evaluation? 

The  Data  Sources  and  Analyses.  Each  of  the  above  questions  will 
be  discussed  in  the  following  paragraphs  with  regard  to  the  analysis 
requirements  and  the  data  sources. 

a.  What  is  the  demonstrated  relative  training  effectiveness  of 


the  BT33  in  training  for  live-fire  FO  missions? 


The  data  sources  for  this  question  were  the  performance  data  from 
the  three  criterion  events.  The  principal  data  subgroup  was  the  adjust 
fire  mission-grid  coordinates  method,  as  this  is  the  most  frequent  mis- 
sion and  method  combination  employed . Answers  to  this  question  will 
consist  of  seven  RE^  values,  one  for  each  of  the  seven  experimental 
conditions.  REfc  is  a measure  of  the  relative  training  effectiveness  of 
the  experimental  condition  as  conpared  to  the  current  training  condi- 
tion. A value  of  REfc  = 1.0  implies  that  the  kth  experimental  condition 
trains  as  effectively,  but  no  more  so,  as  the  current  training  program. 

The  REfc  values  are  an  average  of  the  elements  RE^jk  where: 


'ijk  E. 


*It  should  be  noted  that  the  relative  effectiveness  measure  suggested  by 
TRADOC  (TRADOC  Pam  71-8,  Analyzing  Training  Effectiveness,  Headquarters , 
U.S.  Army  Training  and  Doctrine  Command,  Fort  Monroe,  Va.,  1975)  is  E 
divided  by  C,  which  is  appropriate  when  the  measures  of  E and  C are  such 
that  a larger  value  means  better  performance.  Zn  this  study,  however, 
all  seven  of  the  performance  measures  are  defined  such  that  a smaller 
value  is  better.  This  being  the  cue,  C divided  by  E was  selected  as 
the  relative  effectiveness  measure  so  that  the  effectiveness  ratio  would 
yield  values  consistent  with  usual  expectations  (less  than  1.0  is  a 
worse  condition,  greater  than  1.0  is  a better  condition). 


where : 
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C = the  mean  criterion  performance  score  resulting  from  the 
standard  method  of  FO  training,  experimental  condition 
no.  8; 

E * the  mean  criterion  performance  score  resulting  from  any 
one  of  the  BT33  training  conditions,  experimental  condi- 
tions no.  1-7; 

and  i = performance  measure  i of  measure  nos.  1-7; 

j = criterion  event  j,  with  Mobile  Shoot  One  = 1,  Bunker 
Shoot  = 2,  and  Mobile  Shoot  Two  = 3;  and 

k = experimental  condition  no.  k of  conditions  nos.  1-7. 

The  REjifc  values  combine  across  i and  j to  form  the  seven  RE^ 
values  as  follows: 
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b.  What  is  the  judged  relative  training  effectiveness  of  the  BT33 
in  training  for  live-fire  FO  missions? 

The  data  were  responses  to  Student  Questionnaire  items  1,  2a,  2b, 

3,  and  4;  and  Instructor  Questionnaire  items  1 and  4c-4f.  The  analysis 
principally  consisted  of  an  examination  of  the  means  and  standard  devi- 
ations of  the  rating  responses  and  an  evaluation  of  the  written  responses. 
Also  two-way  4x7  least  squares  ANOVAs  were  run:  the  four  aptitude  levels 
by  the  seven  experimental  conditions  wherein  the  students  trained  on  the 
BT33.  However,  due  to  missing  data,  not  all  experimental  conditions  had 
samples  from  the  four  aptitude  categories.  The  five  or  six  missing  cells 
appeared  r andean  in  nature.  The  SPSS  computer  program  option  chosen  for 
testing  the  main  effects  consisted  of  fitting  the  joint  effects  of  both 
factors  and  then  removing  any  effect  which  might  be  attributable  to  the 
other  factor.  The  results  of  these  analyses  were  used  to  address  both 
this  question  and  the  following  one. 

c.  Are  the  training  programs  differentially  effective  for  students 
of  different  aptitude  levels? 

The  data  included  all  performance  data  on  criterion  events  and 
questionnaire  rating  response  data.  The  ANOVAs  conducted  on  these 
data  were  described  under  the  questions  above. 
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d.  How  well  is  the  BT33  designed  from  a human  factors 
standpoint? 


The  data  are  responses  to  Instructor  Questionnaire  items  8 and 
9;  and  Operator  Questionnaire  items  1-8.  The  means  and  standard  devi 
ations  of  the  rating  responses  were  examined. 


e.  How  well  does  the  BT33  facilitate  instruction  as  c 


training  events  consisting  of  live  fire? 


red  to 


The  data  are  performance  data  on  measure  4,  Total  Mission  Time, 
and  the  Instructor  Evaluation  Questionnaire,  items  6 and  7.  The  total 
mission  times  for  training  events  taking  place  in  the  BT33  were  com- 
pared to  training  event  total  mission  times  on  the  range  to  determine 
if  a time  difference  existed.  The  means  and  standard  deviations  of  the 
rating  responses  were  examined. 

f.  To  what  extent  do  students  and  instructors  want  to  use  the 


BT33  for  training  purposes? 


The  data  sources  are  the  Instructor  Evaluation  Questionnaire, 
item  2,  and  the  Student  Evaluation  Questionnaire,  item  6.  The  means 
and  standard  deviations  of  the  rating  responses  were  examined. 

g.  Can  the  BT33  be  useful  in  student  evaluation? 

The  data  sources  are  the  Instructor  Evaluation  Questionnaire, 
item  5,  and  the  Student  Evaluation  Questionnaire,  item  4d.  The  written 
responses  were  tabulated  and  categorized. 

Special  Data  Adjustments.  It  was  noted  in  an  earlier  section  that 
performance  scores  on  the  Mobile  Shoots  did  differ  as  a function  of 
range  difficulty  and  that  the  difference  between  ranges  would  be  con- 
trolled statistically.  Data  obtained  for  individuals  performing  either 
Mobile  Shoot  on  the  East  range  were  adjusted  to  conform  to  that  obtained 
for  individuals  actually  performing  the  same  Mobile  Shoot  on  the  West 
range.  The  adjustment  was  made  separately  for  Adjust  Fire  Missions  and 
for  Immediate  Suppression  missions,  and  for  each  shoot.  The  adjustment 
process  consisted  of  two  steps:  (1)  identification  of  actual  differ- 
ences in  performance  through  examination  of  means  and  standard  devia- 
tions of  scores  (for  all  variables)  obtained  by  students  on  each  range 
for  each  shoot  by  mission  type?  and  (2)  transforming  all  East  range 
scores  (for  all  variables)  to  the  equivalent  of  a West  range  score 
through  a modified  z-score  transformation  process.  The  formula  is: 


Where: 


individual  score  on  East  range 


and 


= translated  (West)  equivalent  score 

“ijk 

x = mean 

s = standard  deviation 

i = individual  score 

j = performance  measure 

k = mission  type. 


Performance  Data 


RESULTS 


Data  availability  was  analyzed  for  the  four  procedural  conditions, 
based  on  mission  type  and  location  method,  identified  in  the  Design 
section.  It  was  found  that  only  two  combinations  were  used  consistently 
enough  to  provide  N's  sufficiently  large  for  analysis.  Approximately  60 
percent  of  the  data  was  obtained  for  the  Adjust  Fire  mission  using  Grid 
Coordinates  for  location.  This  data  set  was  fully  usable  for  analysis 
among  experimental  conditions  and  only  these  data  are  reported  here. 

Data  obtained  for  the  Immediate  Suppression  mission  using  the  Grid  Co- 
ordinates for  location  provided  a somewhat  less  adequate  sample.  Several 
experimental  conditions  had  no  data  obtained  under  this  combination. 
Summary  results  of  this  data  subgroup  are  presented  in  appendix  B to 
this  report.  The  other  two  procedural  combinations  occurred  so  rarely 
in  the  test  situation  that  data  for  these  conditions  could  not  be 
analyzed. 

The  computer  raw  data  printouts  on  all  of  the  measures  were  examined 
and  found  to  be  reasonable  except  for  the  Accuracy  of  Final  Fire  for  Ef- 
fect Rounds  measure.  The  data  on  this  measure  had  been  reduced  improp- 
erly. Hence  this  measure  was  dropped  from  the  analyses. 

ANOVA  and  ANCOVA  Results.  The  results  of  the  ANOVAs  are  listed 
in  Table  1.  Seeing  that  aptitude  was  a significant  factor  in  seven  of 
the  analyses  and  the  interaction  between  aptitude  and  experimental  con- 
dition was  significant  in  six,  ANCOVAs , using  the  actual  STEP  score  as 
the  covariate,  were  performed.  The  results  of  these  analyses  appear  in 
Table  2. 

In  reviewing  Table  2,  it  can  be  seen  that  aptitude  is  significantly 
related  to  the  performance  measures  as  follows: 


vVx 


ry  of  Significant  Findings  Resulting  from  ANOVAs  on  the  Adjust 
Fire-Grid  Coordinates  Data 


•Covariate  Aptitude. 

* "Experimental  Condition  factor. 

***A  blank  means  that  p for  CovA  > .100  or  p for  EC  > .050,  as  appropriate 


Covaries  on  four  events;  Total  Mission  Time. 

Covaries  on  two  events:  Time  to  Locate  Target  and  Procedural 
Errors  Cut  Score. 

Covaries  on  one  event:  Accuracy  of  Initial  Target  Location  and 
Total  Cuts  Score. 

Covaries  on  no  events:  Number  of  Rounds  Fired. 

This  means  that  for  these  events  the  individual  performance  measure 
scores  were  partially  dependent  upon  the  aptitude  level  (the  STEP  score) 
of  the  student.  On  further  examination  of  the  relationship,  however , 
no  linear  trends  were  observed.  On  some  events  and  for  some  measures 
students  with  high  STEP  scores  did  better  them  those  with  low  scores. 

On  other  events  and  other  measures  the  reverse  was  true.  Also,  in  some 
cases,  the  performance  of  students  scoring  in  the  mid  ranges  was  better 
than  that  of  either  high-  or  low-scoring  students.  In  short,  there 
were  no  definitive  trends  in  the  data.  There  is  no  indication  of  con- 
sistent differential  effectiveness  of  these  training  programs  as  a func- 
tion of  student  aptitude. 


The  most  important  occasions  where  experimental  condition  was  a 
significant  factor  are  the  criterion  events.  This  is  for  two  reasons. 
One,  because  differences  between  conditions  in  training  events  were 
sometimes  primarily  a function  of  field  vs.  simulator  situational  dif- 
ferences (as  in  total  mission  time).  And,  two,  because  the  principal 
matter  of  concern  here  is  whether  the  differences  between  the  experi- 
mental groups,  and  hence  the  REfc  value  differences,  are  significant. 
Looking,  therefore,  at  only  the  criterion  event  results  in  Table  2,  we 
see  that  performances  varied  significantly  as  a function  of  experimental 
condition  in  only  3 cases  out  of  18s  twice  on  the  Total  Mission  Time 
measure  and  once  on  the  Total  Cuts  Score. 

Mean  Scores  and  RE  Values.  There  are  two  possible  sets  of  mean 
performance  scores  per  measure:  (1)  means  based  on  the  raw  data  and 
(2)  means  that  have  been  adjusted  for  the  effect  of  the  covariate,  ap- 
titude. Tables  3 through  8 present  the  subject  Ns  and  mean  scores  based 
on  the  raw  data  for  the  Adjust  Fire  Mission-Grid  Coordinates  method  data 
subgroup.  Tables  9 through  13  present  the  covariate  adjusted  means  for 
those  measures  and  event  combinations  where  the  covariate  was  signifi- 
cant at  the  .10  level  or  better. 

So  as  to  provide  the  reader  with  at  least  one  graphic  view  of  the 
results.  Figure  1 was  developed  for  the  Time  to  Locate  Target  measure. 
The  raw  data  means  or  the  adjusted  means  were  used  as  appropriate,  using 
Tables  3 and  9.  In  viewing  this  graph,  the  overall  learning  trend  is 
apparent  between  Mobile  Shoot  One  and  Two,  as  is  the  comparative  ease 
of  the  Bunker  Shoot.  One  thing  that  can  be  observed  in  this  graph,  and 
also  occurred  on  some  other  measures,  is  change  in  relative  position  of 
the  students  who  had  experimental  condition  7 (all  BT33  substitution) 


Locate  Target  (Seconds)  Raw  Data  Means  by  Event  and  Condition 
(Adjust  Fire-Grid  Coordinates  Data  Subgroup) 


♦Subject  N (N  variations  primarily  due  to  mission-target  location  method  choices) 
•♦Mean  performance  score. 


Mission  Time  (Seconds)  Raw  Data  Means  by  Event  and  Condition 
(Adjust  Fire-Grid  Coordinates  Data  Subgroup) 


♦Subject  N. 

♦♦Mean  performance  score 


Accuracy  of  Initial  Target  Location  (Radial  Error,  Meters)  Raw  Data  Means  by  Event 
and  Condition  (Adjust  Fire-Grid  Coordinates  Data  Subgroup) 


•Subject  N. 

••Mean  performance  score 


of  Rounds  Fired  Raw  Data  Means  by  Event  and  Condition  (Adjust 
Fire-Grid  Coordinates  Data  Subgroup) 


♦Subject  N. 

♦♦Mean  performance  score 


Total  Cuts  Score  Raw  Data  Means  by  Event  and  Condition  (Adjust 
Fire-Grid  Coordinates  Data  Subgroup) 


♦Subject  N. 

♦♦Mean  performance  score 


(XXX)  8# 


Mission  Time  (Seconds)  Aptitude  Covariate  Adjusted  Data  Means  by  Event 
and  Condition  (Adjust  Fire-Grid  Coordinates  Data  Subgroup) 


♦Subject  N. 

♦♦Mean  performance  score 


Procedural  Errors  Cut  Score  Aptitude  Covariate  Adjusted  Data  Means  by  Event  and 
Condition  (Adjust  Fire-Grid  Coordinates  Data  Subgroup) 


•Subject  N. 

••Mean  performance  score 


#8  (XXX) 


across  criterion  events.  On  Mobile  Shoot  One  they  were  the  poorest 
performing  group,  on  the  Bunker  Shoot  they  were  intermediate,  and  on 
the  final  Mobile  Shoot  Two  they  were  the  best  performing  group.  This 
same  trend  did  not  occur  for  all  measures  or  for  all  experimental 
conditions.  Examination  of  Tables  3 through  13  shows  that  Time  to 
Locate  Target  and  Accuracy  of  Initial  Target  Location  did  decrease 
from  Mobile  Shoot  One  to  Mobile  Shoot  Two.  However,  Total  Mission 
Time  (Table  4)  showed  an  increase  over  these  same  shoots;  while  the 
Number  of  Rounds  Fired  (Table  6) , Procedural  Errors  (Table  7) , and 
Total  Cuts  Scores  (Table  8)  showed  mixed  changes  by  condition  with  the 
scores  being  closely  similar  overall  from  the  first  to  the  second  Mo- 
bile Shoot. 


It  was  hypothesized  that  one  advantage  of  the  BT33  over  the  live- 
fire  situation  may  be  a reduction  in  time  needed  per  mission.  The 
live-fire  shoot  was  thought  to  take  longer  due  to  the  delayed  response 
from  the  FDC.  If  this  is  true  then  either  more  students  could  shoot 
more  missions  per  training  period  in  the  BT33,  and  thus  receive  more 
practice,  or  the  training  time  could  be  reduced,  resulting  in  a savings. 
The  mean  total  mission  times  per  BT33  versus  live-fire  sessions  were 
computed  for  the  training  events  and  it  was  found  that  the  anticipated 
time  saving  for  the  BT33  missions  was  indeed  realized:  8.5  minutes 
versus  15.2  minutes,  a difference  of  6.7  minutes  or  44%. 

RE^jjj  values  were  calculated  using  either  raw  data  means  or  co- 
variate adjusted  means  according  to  the  rule:  If  the  covariate  was 
significant  at  the  .100  level  or  better  then  the  adjusted  means  were 
used;  otherwise  the  raw  data  means  were  used.  These  RE^j^  values  are 
presented  in  appendix  C.  The  overall  REj^,  RE j , and  RE*  values  are 
summarized  in  Table  14.  Given  that  experimental  condition  generally 
did  not  account  for  a significant  amount  of  the  performance  variance, 
small  differences  between  these  values  are  not  important . 


Review  of  Table  14  brings  out  some  overall  trends  of  interest  with 
respect  to  each  of  the  RE  indices.  A close  examination  of  the  REj^ 
values  discloses  that  the  relative  position  of  conditions  changes  in 
many  cases  across  events.  For  example,  experimental  conditions  3 and  4 
(XXS  and  SSX)  have  the  highest  ratings  on  Mobile  Shoot  One  and  lower 
relative  positions  on  Mobile  Shoot  Two.  Conditions  5 and  6 (SXS  and 
XSS) , on  the  other  hand,  occupy  lower  relative  positions  on  Mobile  Shoot 
One  and  higher  relative  positions  on  Mobile  Shoot  Two.  Which  simply 
says  that  the  condition  which  rer  1 ‘ in  the  best  initial  live-fire 

ily  the  one  which  results  in 


* t the  REj  va  ^es,  it  earn 
’ons  mcorp^  rating  the  BT33 
.rent  progr  bile  Shoot 


graded  event  performance  is  not  i 
the  best  ultimate  performance.  Ii. 
be  noted  that,  overall,  experimenta1 
result  in  performance  equivalent  t 
One  (REj  « .98)  and  better  them  the  current  program  ir  B inker  Shoot 
and  Mobile  Shoot  Two  (REj  * 1.20,  1.18).  With  rega  E^  values, 

the  overall  order  of  the  experimental  conditions  witn  :t  to  train- 

ing effectiveness  is: 


It  can  be  seen  that  the  RE^  ’ 
tions  4 and  5 being  the  best 


values  fall  into  three  groups,  with  condi - 
, and  conditions  2 and  8 being  the  poorest 


Experimental 

condition 


Mobile 
Shoot  One 


It  is  again  pointed  out  that  the  ANCOVAs  determined  that  the  dif- 
ferences between  performance  as  a function  of  experimental  condition 
were  significant  in  only  3 cases  out  of  18.  Hence,  although  the  above 
REj  and  RE*  values  are  of  interest,  the  differences  between  them  are  of 
negligible  importance . Given  the  lack  of  statistical  significance  in 
the  performance  differences,  the  cost  savings  to  be  realized  from  use  of 
PO  training  devices  is  of  greater  importance . 


Questionnaire  Data 


FAOBC  Student  Evaluation.  This  questionnaire  is  contained  in 
appendix  A.  Student  responses  are  discussed  in  the  following  para- 
graphs. Mean  responses  to  rating  questions  are  listed  in  Table  15. 

The  data  on  each  questionnaire  item  were  analyzed  (ANOVA)  to  determine 
if  either  experimental  condition  or  aptitude  determined  the  responses. 
Where  either  a main  effect  or  interaction  influence  was 'significant, 
this  will  be  discussed. 

a.  FAOBC  Questionnaire  items  1 and  2. 

The  first  four  rating  questions,  contained  in  questionnaire  items 
1 and  2,  asked  the  student  to  evaluate  the  effectiveness  of  the  BT33 — 
both  overall  and  for  specific  clusters  of  skills.  It  is  clear  from 
the  mean  ratings  that  while  the  device  is  rated  overall  as  moderately 
effective  (x  = 3.2),  it  is  seen  to  be  most  effective  as  a procedures 
and  communications  skills  trainer  (3T  = 3.9,  3.7)  and  somewhat  less  ef- 
fective as  a visual/perceptual  skills  trainer  (x  = 2.4). 

The  basis  for  these  ratings  is  spelled  out  in  the  responses  to 
item  2b  concerning  differences  between  performance  in  the  BT33  vs.  on 
the  range.  Nineteen  percent  indicated  procedural  differences  exist  and 
13%  indicated  communications  differences;  but  71%  felt  that  visual/ 
perceptual  differences  exist. 

If  the  FAOBC  students  indicated  that  a difference  did  exist,  they 
were  to  describe  the  difference,  the  duration  of  its  impact  on  range 
performance,  and  suggestions  for  improvement.  The  differences  noted 
primarily  revolved  around  visual/perceptual  problems  with  the  BT33. 

The  problems  included  difficulty  in  range  estimation,  lateral  shift 
estimation,  height  of  burst  (HOB)  estimation,  and  rounds  spotting  in 
the  BT33  with  a short  term  impact  on  range  performance.  The  suggestions 
included  improvement  of  the  film  quality,  the  inclusion  of  pictures  of 
Fort  Sill  terrain  (as  opposed  to  the  shots  of  European  terrain  supplied 
by  SAAB-Scania) , and  the  possibility  of  3-D  projection. 

Although  few  students  noted  any  differences  with  respect  to  com- 
munications skills,  some  who  did  felt  that  the  instructors  were  not 
being  sufficiently  strict  in  the  BT33  and  were  allowing  incomplete 
transmissions  in  the  BT33  classroom  environment.  Others  suggested  the 
addition  of  radios  or  phones  to  the  BT33  simulation. 

ANOVAs  on  the  four  rating  questions  in  items  1 and  2 determined 
that  experimental  condition  was  a significant  response  determining 
factor  on  three  of  these.  The  individual  means  per  condition  for  these 
three  questions  are  presented  in  Table  16. 
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Table  15 


FAOBC  Student  Evaluation  Questionnaire 
Grand  Means  (N's  = 138-278) 


Questionnaire  item 


1.  Overall  BT33  training  effectiveness 


2. 


BT33  effectiveness: 
BT33  effectiveness: 
BT33  effectiveness: 


Procedural  skills  training 
Communication  skills  training 
Visual/perceptual  skills  training 


3. 


Puffboard  vs.  live-fire  effectiveness:  Overall 
BT33  vs.  live-fire  effectiveness:  Overall 


4. 


BT33  vs.  live-fire  effectiveness: 
BT33  vs.  live-fire  effectiveness: 
BT33  vs.  live-fire  effectiveness: 
BT33  vs.  live-fire  effectiveness: 
BT33  vs.  live-fire  effectiveness: 
BT33  vs.  live-fire  effectiveness: 


Puffboard 
Daly  Hill 
Arbuckle  Hill 
Bunker  Shoot 
Mobile  Shoot  One 
Mobile  Shoot  Two 


6.  BT33  availability  for  practice  desirability 


Mean 

rating 


3.2* 


3.9 

3.7 

2.4 


2.0 

2.5 


3.0 
2.5 
2.3 
2.2 

2.0 
2.0 


3.5 


♦Items  rated  on  a 5-point  scale  with  1.0  = very  low  and  5.0  = very  high. 
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Table  16 


Mean  FAOBC  Question  Responses  as  a Function  of 
Experimental  Condition  (N's  = 19-72) 


Experimental  conditions 


FAOBC  questions  (items  1 and  2) 

1 

SXX 

2 

XSX 

3 

XXS 

4 

ssx 

5 

SXS 

6 

XSS 

7 

SSS 

Overall  effectiveness* 

3.0 

3.2 

3.5 

3.1 

3.1 

3.0 

3.4 

Procedural  skills  effective** 

3.7 

4.2 

4.3 

3.8 

3.7 

3.9 

4.3 

Communication  skills 
effectiveness*** 

3.6 

4.0 

3.8 

3.5 

3.7 

3.5 

4.2 

♦Experimental  condition  p = .046,  Aptitude  Interaction  p * .027. 
♦♦Experimental  condition  p = .019. 

♦♦♦Experimental  condition  p = .042. 


In  examining  the  above,  certain  overall  trends  can  be  noted.  The 
students  exposed  to  the  BT33  only  once  (experimental  condition  Mo.  1) 
tended  to  give  low  ratings  to  all  three  questions.  Students  exposed  to 
the  BT33  the  maximum  amount  (condition  No.  7)  tended  to  give  the  high- 
est ratings  on  BT33  effectiveness.  Students  exposed  to  a mix  of  Puff- 
board,  BT33,  and  live-fire  training  exercises  (conditions  2 and  3) 
tended  to  give  moderately  higher  ratings.  Finally,  students  exposed 
to  the  BT33  twice,  with  either  the  Puffboard  or  live  fire  on  the  other 
training  event,  tended  to  give  moderately  lower  ratings. 

The  interaction  effect  was  uninterpretable  due  to  empty  cells. 

It  can  only  be  noted  that  on  the  first  question,  concerning  overall 
BT33  effectiveness,  the  responses  across  the  four  aptitude  categories 
did  interact  with  the  experimental  conditions. 

b.  FAOBC  Questionnaire  item  3. 

The  two  rating  questions  contained  in  this  item  compared  the 
training  effectiveness  of  the  Puffboard  and  the  BT33  to  that  of  a 
live-fire  exercise.  Although  both  were  judged  to  be  somewhat  less 
effective  than  a live-fire  exercise,  the  BT33  received  a higher  rat- 
ing than  did  the  Puffboard  (x  = 2.5  vs.  2.0). 
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Item  4 contained  six  rating  questions,  one  yes/no  question,  and 
a verbal  response  question.  The  rating  questions  (see  Table  IS)  were 
more  specific  versions  of  item  3,  where  the  student  was  asked  to  judge 
the  comparative  effectiveness  of  the  BT33  in  place  of  each  of  the  cur- 
rent training  and  criterion  events.  The  responses  were  as  might  be 
expected — highest  for  the  first  training  event  (x  = 3.0)  and  gradually 
declining  to  smaller  values  on  the  final  criterion  events  (x  = 2.2, 
2.0).  Data  on  three  of  the  six  rating  questions  (item  4b)  were  influ- 
enced significantly  (p  - .040,  .038,  and  .002)  by  an  interaction  be- 
tween the  aptitude  factor  levels  and  the  experimental  conditions.  Due 
however,  to  empty  cells  the  impact  is  uninterpretable. 


The  yes/no  question  under  item  4d  concerned  the  possible  use  of 
the  BT33  for  student  evaluation  purposes:  "Do  you  think  a realistic 
evaluation  of  your  skills  could  be  made  in  the  BT33  rather  than  at 
Bunker  Hill?"  Seventy-seven  percent  of  the  students  responded  "no" 
to  this  question. 


The  verbal  response  question  asked:  "Are  there  any  other  FAOBC 
field  exercises  where  the  BT33  might  be  an  effective  substitute?"  Re 
sponses  pointed  to  the  following: 


The  terrain  association  class,  a dry-fire  shack  shoot,  held 
at  Apache  Ridge; 


Could  be  used  in  conjunction  with  the  FDC  (FAOAC)  exercise; 


• Could  be  a means  of  later  technique  polishing  and  to  maintain 
proficiency ; and 


Should  not  be  used  as  a substitute,  but  could  be  a valuable 
addition,  especially  for  procedural  and  communications  skills 
development. 


d.  FAOBC  Questionnaire  item  5. 

Item  5 asked  for  student  recoomaendations  for  improvement  of  the 
BT33.  The  first  area  covered  (item  5a)  was  terrain,  target,  and  burst 
presentations.  Here  the  students  made  the  following  recommendations, 
with  the  first  two  appearing  most  frequently: 


• Make  the  burst  presentations  longer,  clearer,  and  with  greater 
realism.  When  burst  is  in  woods,  display  a puff  of  smoke. 


Use  displays  of  different  types  of  terrain,  including  shots  of 
difficult  Fort  Sill  terrain,  and  change  them  more  frequently 
during  training. 


• Improve  the  sharpness  of  the  terrain  display,  but  provide 
fewer  terrain  features. 

• Simulate  a mobile  shoot  by  using  different  shots  of  the  same 
target  area. 

• Close  targets  need  to  be  clearer  and  continuously  displayed. 
Target  size  and  clarity  change  would  aid  depth  perception. 

The  second  area  covered  (item  5b)  was  the  manner  in  which  the 
BT33  was  used  by  instructors.  The  most  frequent  responses  here  were: 

(a)  the  instructors  did  a good  job,  (b)  smaller  classes  would  be  very 
desirable  so  that  more  missions  could  be  run  for  students,  and  (c)  which 
students  had  performed  a mission  needed  to  be  monitored  more  carefully 
in  that  not  all  students  received  an  opportunity  to  perform  during  each 
event.  Some  students  also  urged  that  greater  emphasis  be  placed  on 
developing  specific  procedures,  communications,  and  map  reading  skills. 

The  final  question  under  this  item  (5c)  was  an  open-ended  one, 
asking,  "Anything  else?"  The  majority  of  the  comments  made  here  have 
already  been  presented  under  the  above  items  1 through  4.  The  only 
additional  ideas  were:  (a)  the  BT33  should  be  made  available  for  stu- 
dent use  during  evening  hours  and  weekends  and  (b)  the  BT33  was  useful 
in  that  it  allowed  the  instructor  to  train  students  how  to  handle  con- 
tingency situations  like  lost  rounds  and  FDC  errors. 

e.  FAOBC  Questionnaire  item  6. 

Item  6 asked  the  student  to  indicate  the  extent  to  which  he  would 
like  to  have  the  BT33  available  for  additional  practice  outside  the 
presently  scheduled  classes.  The  responses  were  generally  moderately 
favorable,  with  a mean  response  of  3.5. 

These  data  were  influenced  significantly  by  an  aptitude  X experi- 
mental condition  interaction  (p  = .026).  Due,  however,  to  empty  cells 
the  results  are  uninterpretable . 

FAOAC  Student  Evaluation.  The  FAOAC , MASOC,  AIT,  and  III  Corps 
Artillery  questionnaire  is  presented  in  appendix  A.  It  was  responded 
to  only  by  FAOAC  students.  The  mean  FAOAC  student  responses  to  rating 
questions  are  summarized  in  Table  17  and  discussed  in  the  following 
paragraph . 

a.  FAOAC  Questionnaire  items  1 and  2. 

The  FAOAC  students  gavejthe  BT33  fairly  high  ratings  on  training 
effectiveness  both  overall  (x  = 3.5)  and  for  each  of  the  three  sets 
of  skills  (x  = 3.1  to  4.0).  The  ratings  given  by.  the  FAOAC  students 
were  higher  across  the  board  them  those  given  by  the  FAOBC  students 
(compare  Tables  15  and  17) , but  in  agreement  that  the  BT33  is  most 
effective  as  a procedural  and  conmmni cations  skills  trainer  and  least 
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effective  as  a visual  skills  trainer.  These  judgments  of  differential 
effectiveness  per  skill  type  cam  again  be  explained  by  perceived  dif- 
ferences between  visual  performance  in  the  BT33  versus  on  the  ramge. 
Only  12%  perceived  procedural  differences  to  exist  and  only  16%  per- 
ceived communications  differences  to  exist;  whereat  61%  perceived 
visual/perceptual  differences. 


Table  17 

FAOAC  Student  Evaluation  Questionnaire  Mean  Responses 


Questionnaire  item 


Mean 

rating 


1.  Overall  BT33  training  effectiveness  (N  = 142) 


2.  BT33  effectiveness:  Procedural  skills  training  (N  = 142) 
BT33  effectiveness:  Communications  skills  training 
(N  = 142) 

BT33  effectiveness:  Visual/Perceptual  skills  training 
(N  = 142) 


3.5 

4.0 
3.5 

3.1 


Puffboard  vs.  live-fire  effectiveness:  Overall  (N  = 48) 
BT33  vs.  live-fire  effectiveness:  Overall  (N  = 98) 


2.4 

3.1 


4.  BT33  vs.  live-fire  effectiveness:  FAOAC  program  (N  * 127) 
6.  BT33  availability  for  practice:  Desirability  (N  = 111) 


2.9 

2.8 


4 


♦items  rated  on  a 5-point  scale  with  1.0  = very  low  and  5.0  = very  high 


If  the  FAOAC  students  indicated  that  a difference  did  exist,  they 
were  to  describe  the  difference,  the  duration  of  its  impact  on  range 
performance,  and  suggestions  for  improvement.  The  main  differences 
noted  were  the  lack  of  radios  and  3-D  displays.  The  principal  problems 
noted  revolved  around  the  difficulty  in  visually  estimating  ranges  and 
HOBs  on  a 2-D  display  where  neither  burst  nor  target  sizes  changed  as 
a function  of  distance.  The  problem  was  apparently  most  acute  for 
students  seated  close  to  the  screen.  The  students  generally  felt  the 
impact  of  this  perceptual  problem  on  range  performance  was  short-lived. 
They  recommended  improved  film  quality,  a greater  variety  of  pictures, 
and  3-D  projection. 
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b.  FAOAC  questionnaire  items  3 and  4 


The  FAOAC  students  also  gave  higher  ratings  to  the  two  questions 
contained  in  item  3 than  did  the  FAOBC  students  (compare  Tables  15  and 
17) . They  judged  the  Puffboard  to  be  somewhat  less  effective  than  live 
fire  (x  * 2.4),  while  the  BT33  was  judged  to  be  as  effective  overall 
as  live  fire  (x  = 3.1). 


In  judging  the  effectiveness  of  the  BT33  in  contrast  with  live 
fire  for  their  specific  FAOAC  training  program  (question  4a) , the 
students  again_judged  the  BT33  to  be  just  about  as  training  effective 
as  live  fire  (x  = 2.9).  The  responses  to  question  4b,  "How  often  and 
in  what  manner  do  you  think  the  BT33  might  be  best  used  in  the  FAOAC? 
were  grouped  into  the  following  categories: 


• The  BT33  could  be  used  entirely 
eliminating  live  fire 


• The  BT33  could  be  used  "extensively 
"frequently,"  or  "equally" 


• The  BT33  could  be  used  once  or 
twice  for  refresher  training 


• The  BT33  should  not  be  used  at  all 


c.  The  FAOAC  questionnaire  item  5 


Questionnaire  item  5 asked  for  recommendations  for  improvement  of 
the  BT33 . The  first  question  in  this  item  concerned  terrain,  target, 
and  burst  presentations.  The  recommendations  were 


Bursts:  color  them;  scale  them,  making  the  close  in  ones 
larger  than  the  current  ones;  and  display  for  a 
longer  time. 


The  second  question  concerned  the  manner  in  which  the  BT33  is 
used  by  the  instructors.  The  comments  made  here  and  elsewhere  can  best 
be  represented  by  one  of  the  responses:  "Instructors  must  be  sure  that 
target  location  and  orientation  on  the  BT33  are  accurately  set  up.  Poor 
data  inputs  defeat  the  purpose  with  respect  to  student  attitude." 
Granted,  the  BT33  can  be  used  to  introduce  the  students  to  the  falli- 
bilities of  the  FDC  and  gunnery.  But  the  matter  of  how  and  when  to  do 
this  apparently  needs  to  be  reconsidered. 


The  third  question  was  a catchall,  "Anything  else?"  i 
ideas  presented  here  have  already  been  discussed  elsewhere 


Whereas  the  FAOAC  students  gave  higher  ratings  than  did  the  FAOBC 
students  on  all  the  preceding  questions,  they  reversed  the  trend  on 
this  item.  While  the  FAOBC  students  had  indicated  a definite  desire 
to  have  the  BT33  available  for  practice  (3c  = 3.5),  the  more  experience< 
FAOAC  students  did  not  feel  a need  to  quite  the  same  extent  (x  = 2.8). 


Instructor  Evaluation.  The  Insi 
is  presented  in  appendix  A.  The  meat 
ized  in  Table  18  and  discussed  below 


Instructor  Evaluation  Questionnaire  Mean  Responses 


BT33  vs.  range  training  environing] 
How  well  the  student  learns  (N  = : 
How  quickly  the  student  learns  (N 
For  FAOBC  training  (N  = 9) 

For  FAOAC  training  (N  = 5) 

For  AIT  training  (N  = 4) 

For  MASOC  training  (N  = 3) 

For  field  FO  training  (N  = 19) 


6.  BT33  facilitation  of  instructor-student  interaction 


7.  BT33  facilitation  of  student  learning  problems 
diagnosis  (N  = 19) 


8.  Adequacy  of  BT33  presentation  management  (N  * 19) 


*Items  rated  on  a 5-point  scale  with  1.0  ■ very  low  and  5.0  = very  high 


a.  Instructor  Questionnaire  items  1,  2,  and  4 


The  instructors  rated  the  BT33  as  fairly  effective  overall  (item 
1,  x ■ 3.4).  The  FAOBC  instructors  (N  = 9)  indicated  that  22%  of 
FAOBC  training  could  take  place  in  the  BT33  while  the  FAOAC  instructors 
(N  = 5)  indicated  20%  for  FAOAC  training  (item  2) . 

Item  4 asked  the  instructors  to  make  judgments  regarding  student 
learning  behavior  and  the  effectiveness  of  the  BT33,  as  opposed  to  live 
fire,  for  specific  training  programs.  Two  of  the  items  in  Table  18, 

BT33  effectiveness  for  FAOAC  training  and  for  field  FO  training,  were 
given  rather  low  ratings  and  hence  warrant  comment.  The  low  rating 
given  by  five  instructors  of  BT33  effectiveness  for  FAOAC  students 
(x  = 1.6)  seems  to  be  in  conflict  with  the  moderately  high  rating  given 
by  the  FAOAC  students  themselves  (Table  17,  x = 2.9) — but  it  may  not  be. 
The  majority  of  the  FAOAC  students  also  indicated  that  the  BT33  would 
be  most  useful  for  one  or  two  FAOAC  sessions  but  not  much  beyond  that. 
This  may  be  just  what  the  instructors  were  saying  as  well. 

The  rating  given  with  regard  to  field  FO  training,  x = 2.2,  is  a 
more  real  rating  in  that  it  is  based  on  a larger  N of  19.  It  is  also 
more  of  a problem  in  that  the  major  objective  of  the  OFT  now  under  de- 
velopment is  to  provide  this  unit,  or  field,  FO  training.  It  may  be 
that  the  instructors  were  actually  rating  the  BT33  with  respect  to  its 
lack  of  suitability  for  the  more  rugged  unit  environment  rather  than 
with  respect  to  its  training  value.  On  the  other  hand,  the  ratings  may 
indicate  a real  doubt  regarding  the  value  of  a FO  unit  training  device. 
If  so,  then  the  OFT  CTEA  may  need  to  be  extended  to  the  unit  environment 
with  the  LRIP  models  prior  to  initiating  full  rate  production  of  the 
OFT.  Certainly  the  opinions  of  the  instructors  regarding  the  value  of 
a FO  trainer  in  the  unit  environment  need  to  be  reexamined. 


b.  Instructor  Questionnaire  items  5 through  8. 


Items  5 through  8 dealt  with  questions  related  to  student  manage- 
ment in  the  BT33  environment.  The  first  of  these,  item  5,  concerned 
the  issue  of  whether  student  performance  could  be  evaluated  in  the 
BT33.  Forty-seven  percent  of  the  instructors  felt  that  the  answer  was 
Y'  i.  These  instructors  generally  felt  that  this  evaluation  should 
take  place  early  in  training  to  evaluate  procedural  learning  and 
progress . 


With  regard  to  instructor  capabilities  to  interact  with  students 
and  diagnose  their  learning  problems,  the  instructors  seemed  to  feel 
that  the  BT33  and  range  environments  were  about  the  same  (x  = 3.0  and 
2.9). 
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Finally,  with  regard  to  instructor  capability  to  manage  BT33 
display  presentations,  the  instructors  on  the  whole  felt  fairly  well 
in  control.  The  mean  response  to  question  8 was  3.4,  with  90%  indi- 
cating that  no  specific  problems  exist. 


Operator  Evaluation.  This  questionnaire  is  presented  in  appen- 
dix A.  Operator  responses  are  discussed  in  the  following  paragraphs 


a.  Questionnaire  items  1 through  4 


The  first  four  items  asked  the  operator  for  ratings,  yes/no  re- 
sponses, and  opinions  regarding  the  operability  of  the  BT33.  The  mean 
ratings  for  items  1 through  3 are  given  in  Table  19.  In  discussing 
the  issue  of  how  easy  it  is  to  damage  the  equipment,  it  was  pointed 
out  that  the  target  and  sound  track  tapes  were  very  easy  to  damage  or 
tear,  and  that  projector  fan  belts  and  firing  battery  buttons  came 
apart  easily. 


* I terns  1 through  3 rated  on  a 5-point  scale  with  1.0 
and  5.0  = best  rating  possible. 


••Items  5 and  6 rated  on  a 5-point  scale  with  1.0 
5.0  a a great  dead. 


In  responding  to  item  4,  "Are  there  any  other  specific  problems 
in  the  layout  or  operation  of  the  device?,"  two  of  the  three  operators 
answered  yes.  The  problems  mentioned  were  that  the  unit  should  be 
more  compact  and  that  the  projectors  were  difficult  to  service  and 
align  due  to  location. 


b.  Questionnaire  items  5 and  6. 

Items  5 and  6 concerned  the  amount  of  electronic  equipment,  and  FO 
background  required  to  become  a BT33  operator.  As  can  be  seen  from 
Table  19,  the  operators  felt  that  little  electronic  background  was 
needed,  but  that  a moderate  FO  procedures  and  map  reading  background 
would  be  helpful.  Comments  were  to  the  effect  that  a 6 months’  back- 
ground in  the  13E  MOS  was  desirable. 

c.  Questionnaire  items  7 and  8. 

These  items  address  the  questions  of  how  much  training  and  prac- 
tice is  needed  to  become  an  operator  and  then  maintain  proficiency. 

The  operators  indicated  t)iWt_2  to  5 days  (x  = 3)  classroom  training 
was  needed  and  1 to  5 days  (x  = 4)  OJT.  Two  of  the  operators  thought 
that  once  a month  operation  was  sufficient  to  maintain  proficiency, 
while  the  third  operator  felt  that,  given  2 months'  experience  on  the 
equipment  first,  once  a week  was  necessary. 


DETAILED  CONCLUSIONS 

The  conclusions  of  the  study  are  discussed  in  terms  of  the  ques- 
tions presented  in  the  Analysis  Procedures  section. 

a.  What  is  the  demonstrated  relative  training  effectiveness  of 


the  BT33  for  live-fire  FO  missions? 


The  differences  in  performance  on  the  criterion  events  did  not, 
except  in  3 cases  out  of  18,  significantly  differ  as  a function  of  ex- 
perimental condition  (see  Table  2).  Given  that,  it  can  be  said  that, 
overall,  the  seven  experimental  training  programs  with  the  BT33  are  as 
good  as  the  current  training  program.  Experimental  conditions  4 and  5 
(SSX  and  SXS)  tend  to  produce  slightly  better  training  overall  (REk  = 
1.20  and  1.19,  while  experimental  conditions  2 and  8 (XSX  and  XXX,  the 
current  program)  tend  to  produce  the  poorest  training  overall  (RE^  = 
.98  and  1.00)  (see  Table  14).  Programs  6 anc'.  7 (XSS  and  SSS)  tend  to 
be  intermediately  effective  (RE*  = 1.10  and  1.12)  and  are  the  most 
cost  effective  due  to  the  reduction  of  live-fire  and  ammunition  costs. 

b.  What  is  the  iudged  relative  training  effectiveness  of  the 


BT33  for  live-fire  FO  missions? 


The  FAOBC  and  FAOAC  students  gave  the  BT33  fairly  high  ratings 
for  early  procedures  and  communications  skills  training  (see  Tables  15 
and  17) . The  BT33  was  rated  as  somewhat  less  them  moderately  effec- 
tive for  visual/perceptual  skills  training,  however,  and  short-term 
problems  were  noted  upon  transition  to  the  field  environment.  This 
visual  problem  may  be  the  reason  for  the  lower  RE  value  obtained  on 
Mobile  Shoot  One  (see  Table  14) . The  fact  that  the  RE  values  for  the 
final  two  graded  events  were  higher  substantiates  the  judgments  that 


the  transition  problems  were  indeed  short-lived  and  negates  the  im- 
portance of  the  problem  given  some  field  training  in  live  fire. 

The  instructors  judged  the  BT33  to  be  moderately  effective  over- 
all and  for  three  groups  of  students:  FAOBC,  AIT,  and  MASOC  (see 
Table  18) . They  judged  it  as  less  than  moderately  effective  for  the 
FAOAC  and  unit  students.  The  implications  of  the  latter  two  exceptions 
for  OFT  are  not  clear  for  two  reasons: 

1.  The  FAOAC  students,  in  contrast  to  their  instructors,  gave 
the  BT33  even  higher  ratings  than  did  the  FAOBC  students. 

This  difference  of  opinion,  based  on  a very  small  N in  the 
case  of  the  instructors  (N  = 5),  may  give  greater  weight  to 
the  higher  rating  given  by  the  students. 

2.  The  BT33  is  not  designed  for  the  field  environment,  whereas 
OFT  will  be. 

It  is  not  known  whether  the  instructors  were  rating  the  BT33  in  terms 
of  its  field  environment  suitability  or  in  terms  of  its  likely  unit 
training  effectiveness. 

The  conclusions  to  be  derived  here  are  two.  One,  the  BT33  is 
judged  to  be  effective  in  the  institutional  training  environment,  so 
long  as  the  need  for  adequate  live-fire  training  continues  to  be  recog- 
nized. Two,  the  instructors  should  be  assessed  again  regarding  their 
views  concerning  the  value  of  the  OFT  in  the  unit  environment.  If 
doubts  really  exist  then  it  might  be  well  to  extend  the  OFT  CTEA  evalu- 
ation to  the  unit  environment  when  the  LRIP  OFT  models  are  received. 


c.  Are  the  training  proqrams  differentially  effective  for  stu- 


dents of  different  aptitude  levels? 


Results  of  the  ANOVAs  indicate  that  the  eight  experimental  pro- 
grams were  differentially  effective  based  on  aptitude  differences  for 
7 out  of  30  data  points.  The  interaction  between  experimental  condi- 
tion and  aptitude  was  significant  on  three  out  of  five  events  for  the 
Total  Cuts  Score,  two  events  for  Total  Mission  Time,  and  one  event  for 
Accuracy  of  Initial  Target  Location  (see  Table  1) . The  significant 
interactions  show  that  the  individual  programs  produced  different  per- 
formance results  for  subjects  of  high  versus  low  aptitudes,  but  the 
relationships  between  the  STEP  score  and  performance  were  not  consistent 

A more  detailed  examination  was  also  conducted  to  determine  if 
the  BT33  is  itself  differentially  effective  for  different  aptitudes  as 
compared  to  the  current  training  environments.  The  relationships  be- 
tween STEP  score  and  performance , as  assessed  by  the  correlation  coef- 
ficient, were  examined  for  each  individual  training  event  x experimental 
condition  situation.  Again,  no  consistent  relationships  were  found. 


d.  How  well  is  the  BT33  designed  from  a human  factors  standpoint? 


The  operator  and  instructor  judgments  indicated  that  the  BT33  is 
designed  fairly  well  from  an  operability  standpoint  but  not  so  well 
from  a maintainability  standpoint  (see  Tables  18  and. 19).  Specific 
problems  are  discussed  on  pp.  39-41. 

e.  How  well  does  the  BT33  facilitate  instruction  as  compared  to 


training  events  consistinq  of  live  fire? 


The  BT33  appears  to  facilitate  instruction  at  least  moderately 
well  in  terms  of  student  learning  speed,  instructor-student  interac- 
tions, and  diagnosis  of  student  learning  problems  (see  Table  18).  Fur- 
ther, it  was  determined  that  a BT33  mission  takes  an  average  of  8.5 
minutes,  as  compared  to  15.2  minutes  for  a live-fire  mission;  i.e.,  it 
takes  44%  less  time  to  shoot  a mission  in  the  BT33  than  in  the  field. 
Hence  either  students  can  shoot  more  missions,  thus  receiving  more 
practice,  or  the  class  time  could  be  cut.  The  latter  option  should 
not  be  employed  however  without  careful  testing  to  insure  that  adequate 
training  would  still  be  received. 

f.  To  what  extent  do  students  and  instructors  want  to  use  the 


BT33  for  training  purposes? 


Based  on  their  comments  and  evaluations  regarding  training  ef- 
fectiveness, plus  the  desire  evidenced  by  both  the  FAOBC  and  FAQAC 
students  for  additional  time  on  the  BT33  outside  of  class  (see  Tables 
15  and  17) , it  appears  that  the  majority  of  students  do  want  to  use  the 
BT33  to  at  least  some  extent.  The  instructors  indicated  that  20%  to 
22%  of  the  FO  training  could  well  take  place  in  the  BT33. 

Overall,  it  appears  that  most  students  will  accept  the  BT33  at 
least  to  the  extent  that  the  instructors  do.  The  principal  concern 
of  the  students  is  obtaining  a high  score  in  the  graded  events.  If 
they  are  reassured  that  their  overall  grade  average  will  be  as  high  if 
they  use  the  BT33,  using  the  findings  of  this  study  as  evidence,  even 
more  students  will  accept  the  BT33  positively. 

. Can  the  BT33  be  useful  in  student  evaluation? 
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APPENDIX  A 


QUESTIONNAIRES 


USAFAS  BT33  TRAINING  EFFECTIVENESS  STUDY 
STUDENT  EVALUATION 
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The  USAFAS,  with  the  assistance  of  the  Army  Research 
Institute,  is  evaluating  the  utility  of  the  BT33  Fire  Con- 
trol Simulator  for  Forward  Observer  training.  It  is  a 
Swedish  training  device,  built  by  Saab-Scania.  It  is  com- 
parable in  many  ways  to  the  Army's  own  Observed  Fire 
Trainer,  which  is  now  under  development. 

As  an  aid  in  evaluating  the  impact  of  this  training 
device  on  the  student  and  how  it  might  best  be  used, 
please  fill  out  this  cover  sheet  and  the  attached  question- 
naire as  completely  as  you  can.  Your  answers  will  be  used 
for  research  purposes  only  and  will  not  become  a part  of 
your  record,  when  identifiers  (name  and  Social  Security 
Account  Number)  are  requested  they  are  to  be  used  for  ad- 
ministrative and  statistical  control  purposes  only.  Full 
confidentiality  of  the  responses  will  be  maintained  in 
the  processing  of  these  data. 

Your  participation  in  this  research  is  strictly 
voluntary.  Individuals  are  encouraged  to  provide  complete 
and  accurate  information  in  the  interests  of  this  equip- 
ment evaluation,  but  there  will  be  no  effect  on  indi- 
viduals for  not  providing  all  or  any  part  of  the  infor- 
mation. In  keeping  with  the  Privacy  Act  of  1974,  we 
request  that  you  sign  your  name  at  the  bottom  of  the 
cover  sheet  indicating  that  you  have  not  objected  to 
completing  this  form. 


CLASS  (e.g.,  FAOBC  9-76) 


SECTION 


Last  First 

1.  Time  as  a commissioned  officer  (months) 

2.  Source  of  commission  (check  one):  BIOCC 

ROTC 

SIGNATURE 


. i v M 


In  several  of  the  questions  you  will  oe  asked  to  evaluate  some 
characteristic  of  the  BT33  through  use  of  a rating  scale.  Please 
regard  these  scales  as  being  continuous.  Place  an  nxM  at  what- 
ever location  on  the  scale  best  describes  your  opinion. 


Overall,  how  effective  do  you  feel  the  BT33  is  as  a training 
device? 


Very 

Ineffective 


Moderately 

Effective 


Very 

Effective 


Performance  of  the  Forward  Observer's  job  involves  the  use  of 
procedural,  communications,  and  visual/perceptual  skills.  In 
answering  the  following  questions,  think  about  what  you  learned 
and  the  manner  of  performance  of  each  of  these  in  the  BT33  train- 
ing environment  vs.  out  on  the  range  during  live  firing  exercises. 


a.  How  effective  do  you  feel  the  BT33  is  for  training  each  of 
these  skills: 

Procedural? 


Very 

Moderately 

Very 

Ineffective 

Effective 

Effective 

Conanunications? 

Very 

Moderately 

Very 

Ineffective 

Effective 

Effective 

Visual/Perceptual? 

b.  Wire  there  any  differences  In  the  manner  of  your  performance 
In  the  BT33  vs.  range  environments  which  caused  you  diffi- 
culty when  performing  on  the  range? 

Procedural  differences?  Yes  

= No  

If  the  above  answer  was  "Yes,"  please  describe  the  differences  and 
indicate  whether  the  difficulty  you  experienced  was  very  short- 
term or  lasted  for  some  longer  period  of  time. 
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Communications  differences?  Yes  

No  

If  the  above  answer  was  "Yes,"  please  describe  the  differences  and 
indicate  whether  the  difficulty  you  experienced  was  very  short- 
term or  lasted  for  some  longer  period  of  time. 


Visual/Perceptual  differences?  Yes  

' ' " No  

If  the  above  answer  was  "Yes,"  please  describe  the  differences  and 
indicate  whether  the  difficulty  you  experienced  was  very  short- 
term or  lasted  for  some  longer  period  of  time. 


3.  Overall,  how  do  the  BT33  and  Puff  Board  training  devices  compare 
to  a live  fire  training  exercise  for  training  effectiveness? 

(If  you  were  not  exposed  to  the  Puff  Board  please  skip  that  part 
of  the  evaluation.) 

Puff  Board: 


1 

1 

1 

1 

1 

Much  Less 

Same 

More  More 

Effective 

Effective 

BT33  Training  Device: 

| h- 

1 

1 

1 

Much  Less  Same  Much  More 

Effective  Effective 


4.  Now  more  specifically,  how  does  the  BT33  compare  to  live  fire  ex- 
ercises for  training  effectiveness  at  different  points  in  training? 

a.  The  very  first  practice  session  (known  as  the  "Puffboard"  exer- 
cise) . How  does  the  BT33  compare  to  a live  fire  exercise  for 
training  effectiveness  on  this  session? 

I 1 j 1 1 

Much  Less  Same  Much  More 

Effective  Effective 


b.  Daly  Hill  exercise.  (Compare  the  BT33  to  the  Daly  Hill  situa- 
tion for  training  effectiveness.) 


1 

1 

1 

1 

1 

Much  Less 

Same 

Much  More 

Effective 

Effective 

c . Arbuckle 

Hill  exercise. 

1 

1 

1 

1 

1 

Much  Less  Same  Much  More 

Effective  Effective 
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d.  Bunker  Hill  Shoot. 

I 1 1- 

Much  Less  Same 

Effective 


Much  More 
Effective 


Do  you  think  a realistic  evaluation  of  your  skills  could  be  made 

in  the  BT33  rather  than  at  Bunker  Hill?  Yes  

No 


e.  Mobile  Shoot  One. 


I 

Much  Less 
Effective 


— f— 

Same 


+ 


Much  More 
Effective 


f.  Mobile  Shoot  Two. 

1 1 {— 

Much  Less  Same 

Effective 


Much  More 
Effective 


g.  Are  there  any  other  FAOBC  field  exercises  where  the  BT33 
might  be  an  effective  substitute? 


Do  you  have  any  recommendations  for  improvement  of  BT33  training 
effectiveness  with  respect  to  any  of  the  following? 


a.  Terrain,  target,  or  burst  presentations? 


b.  The  manner  in  which  the  BT33  is  used  by  the  instructors? 


c.  Anything  else? 


Would  you  like  to  have  the  BT33  available  to  you  for  additional 
practice  outside  of  the  presently  scheduled  class  training 
sessions? 


Would 

Dislike 


USAFAS  BT33  TRAINING  EFFECTIVENESS  STUDY 
STUDENT  EVALUATION 


V 


f 


■ 


FAOAC,  MASOC,  AIT  AND  III  CORPS  ARTILLERY 


The  USAFAS.  with  the  assistance  of  the  Army  Research 
Institute , is  evaluating  the-  utility  of  the  BT33  Fire  Con- 
trol Simulator  for  Forward  Observer  training.  It  is  a 
Swedish  training  device,  built  by  Saab-Scania.  It  is  com- 
parable in  many  ways  to  the  Army's  own  Observed  Fire 
Trainer,  which  is  now  under  development. 

As  an  aid  in  evaluating  the  impact  of  this  training  de- 
vice on  the  student  and  how  it  might  best  be  used,  please 
fill  out  this  cover  sheet  and  the  attached  questionnaire  as 
completely  as  you  can.  Your  answers  will  be  used  for  re- 
search purposes  only  and  will  not  become  a part  of  your 
record.  When  identifiers  (name  and  Social  Security  Account 
Number)  are  requested  they  axe  to  be  used  for  administrative 
and  statistical  control  purposes  only.  Full  confidentiality 
of  the  responses  will  be  maintained  in  the  processing  of 
these  data. 

Your  participation  in  this  research  is  strictly  volun- 
tary. Individuals  are  encouraged  to  provide  complete  and 
accurate  information  in  the  interests  of  this  equipment 
evaluation,  but  there  will  be  no  effect  on  individuals  for 
not  providing  all  or  any  part  of  the  information.  In  keep- 
ing with  the  Privacy  Act  of  1974,  we  request  that  you  sign 
your  name  at  the  bottom  of  the  second  cover  sheet  indicating 
that  you  have  not  objected  to  completing  this  form. 


CLASS  (e.g.,  FAOBC  9-76)  SECTION  

DATE  

NAME  SSAN 

Last  First  MI 


FOR  ADP 
USE  ONLY 

1.  

2.  

3.  

4.  

5.  

6.  

7.  

8.  

9.  

10.  

11.  

12.  

13.  

14. 

15.  

16.  

17.  

18.  

19.  

20.  

21.  

22.  

23.  

24.  

25.  

26.  

27.  

28.  

29.  

30.  

31.  

32.  

33.  

34.  

35.  

36. 
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1.  Officer  Enlisted  (check  one) 

2.  Rank  

3.  Time  in  the  military  months 

4.  Prior  enlisted  service  months 

5.  Source  of  commission  (check  one  if  applicable) : BIOCC  USMA 

ROTC  

6.  Have  you  ever  served  as  an  FO?  Yes  No  

How  long  months  In  combat?  Yes No  

7.  How  many  missions  have  you  fired  since  OBC/OCS?  (estimate)  

8.  Which  program  are  you  currently  enrolled  in?  (check  one) 

FAOAC  MASOC  AIT  III  Corps  Artillery  

NOTE:  PLEASE  RESPOND  TO  THIS  QUESTIONNAIRE  REGARDING  THE  BT33  ONLY 
IN  TERMS  OF  ITS  POTENTIAL  USE  WITHIN  THE  PROGRAM  YOU  ARE 
CURRENTLY  IN. 


3 

I 


ii 


SIGNATURE 
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In  several  of  the  questions  you  will  be  asked  to  evaluate  some 
characteristic  of  the  BT33  through  use  of  a rating  scale.  Please 
regard  these  scales  as  being  continuous.  Place  an  "x"  at  what- 
ever location  on  the  scale  best  describes  your  opinion. 


Overall 

device? 


Performance  of  the  Forward  Observer's  job  involves  the  use  of  pro- 
cedural, coanunications , and  visual/perceptual  skills.  In  answer- 
ing the  following  questions,  think  about  what  you  learned  and  the 
manner  of  performance  of  each  of  these  in  the  BT33  training  environ' 
aant  vs.  out  on  the  range  during  live  firing  exercises. 


How  effective  do  you  feel  the  BT33  is  for  training  each  of 
these  skills? 


Visual/Perceptual? 


b.  were  there  any  differences  in  the  manner  of  your  performance 
in  the  BT33  vs.  range  environments  which  caused  you  difficulty 
when  performing  on  the  range? 

Procedural  differences?  Yes  


If  the  above  answer  was  "Yes,"  please  describe  the  differences  and 
indicate  whether  the  difficulty  you  experienced  was  very  short- 
term or  lasted  for  some  longer  period  of  time. 


f 


I 

M 


Communications  differences? 


If  the  above  answer  was  "Yes,"  please  describe  the  differences  and 
indicate  whether  the  difficulty  you  experienced  was  very  short- 
term or  lasted  for  some  longer  period  of  time. 


Visual/Perceptual  differences?  Yes  

No  

If  the  above  answer  was  "Yes,"  please  describe  the  differences  and 
indicate  whether  the  difficulty  you  experienced  was  very  short- 
term or  lasted  for  some  longer  period  of  time. 


3.  Overall,  how  do  the  BT33  and  Puff  Board  training  devices  compare 
to  a live  fire  training  exercise  for  training  effectiveness? 

(If  you  were  not  exposed  to  the  Puff  Board  please  skip  that  part 
of  the  evaluation.) 

/ 


Puff  Board: 
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FAQAC  b 


How  often  and  in  what  manner  do  you  think  the  BT33 
might  be  best  used  in  the  FAQAC? 


How  does  the  BT33  compare  to  live  fire  exercises  for 
training  effectiveness  in  the  AIT  program?  (Answer 
only  if  you  are  or  have  been  in  AIT. ) 


Much  Less 
Effective 


Much  More 
Effective 


How  often  and  in  what  manner  do  you  think  the  BT33 
might  be  best  used  in  the  AIT  program? 


Ill  Corps 
Ar tiller} 


Much  Less 
Effective 


III  Cort 
Artillei 


How  does  the  BT33  compare  to  live  fire  exercises 
for  refresher  training  effectiveness?  (Answer 
only  if  you  are  a member  of  the  III  Corps 
Artillery. ) 


Much  More 
Effective 


How  often  and  in  what  manner  do  you  think  the 
BT33  might  be  best  used  for  refresher  training? 


Do  you  have  any  recommendations  for  improvement  of  BT33  training 
effectiveness  with  respect  to  any  of  the  following? 

a.  Terrain,  target,  or  burst  presentations? 


b.  The  manner  in  which  the  BT33  is  used  by  the  instructors? 


fL 


I 


6 . Mould  you  like  to  have  the  BT33  available  to  you  for  additional 

practice  outside  of  the  presently  scheduled  class  training  sessions? 


i — 

Would 

Dislike 

Very 

Much 


4- 


-4- 


+ 


Moderately 

Like 


Would 

Like 

Very 

Much 


¥ 

k Mi 


U SAFAS  BT33  TRAINING  EFFECTIVENESS  STUDY 
INSTRUCTOR  EVALUATION 

FOR  ADP 
USE  ONLY 


The  U SAFAS,  with  the  assistance  of  the  Any  Research  1. 

Institute  (ARI) , is  evaluating  the  utility  of  the  BT33  Fire  2. 
Control  Simulator  for  Forward  Observer  training.  It  is  a 3. 

Swedish  training  device,  buixt  by  SAAB-Scania.  It  is  com-  4. 
parable  in  many  ways  to  the  Army's  own  Observed  Fire  5. 

Trainer,  which  is  now  under  development.  6. 

7. 

As  an  aid  in  evaluating  the  characteristics  and  utility  8. 
of  this  training  device  as  an  instructional  tool,  please  9. 

fill  out  this  cover  sheet  and  the  attached  questionnaire  as  10. 
completely  as  you  can.  Your  answers  will  be  used  for  re-  11. 
search  purposes  only  and  will  not  become  a part  of  your  12. 

record.  When  identifiers  (name  and  Social  Security  Account  13. 
Number)  are  requested  they  are  to  be  used  for  admini  tra-  14. 
tive  and  statistical  control  purposes  only.  Ful]  rfi-  15. 

dentiality  of  the  responses  will  be  maintained  in  16. 

processing  of  these  data.  17. 

18. 

Your  participation  in  this  research  is  strictly  volon-  19. 
tary.  Individuals  are  encouraged  to  provide  complete  and  20. 
accurate  information  in  the  interests  of  this  equipment  21. 

evaluation,  but  there  will  be  no  effect  on  individuals  22. 

for  not  providing  all  or  any  part  of  the  information.  In  23. 

keeping  with  the  Privacy  Act  of  1974,  we  request  that  you  24. 

sign  your  name  at  the  bottom  of  the  second  cover  sheet  25. 

indicating  that  you  have  not  objected  to  completing  this  26. 

form.  27 . 

28. 

29. 

NAME  _ 


Last  First  MI 

DATE  SSAN  

INSTRUCTOR  POOL  MEMBERSHIP:  FAOBC  FAOAC 


NOTE:  Please  respond  to  this  questionnaire  only  in  terms 
of  your  own  current  teaching  background  (i.e., 
FAOBC  or  FAOAC). 
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i 


‘ 

I 

I 

I 


I 


Months 


Source  of  commission  (check  one) : BIOCC 

ROTC 


USMA 


Months 


Months  In  combat? 


How  many  missions  have  you  fired  since  OBC/OCS  (estimate)? 


Months 


How  many  classes  have  you  handled? 


SIGNATURE 
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How  quickly  the  student  learns  his  skills? 


Much  Less 

Same 

Much  More 

Effective 

Effective 

c.  For  

FAOBC  (check  one)  training? 

FAOAC 

Much  Less 

Same 

Much  More 

Effective 

Effective 

d.  For  AIT  training? 

(Respond  only  if  you  have 

instructed  an  AIT 

class. ) 

Much  Less 
Effective 


Much  More 
Effective 


e.  For  MASOC  training?  (Respond  only  if  you  have  instructed  an 
MASOC  class.) 


■ 

Much  Less 

i ■ • 

Same 

• 

Much  More 

Effective 

Effective 

f.  For  Forward 

• 

Observer  training  in  the  field? 

A la 

! 

Much  Less 
Effective 


Much  More 
Effective 


Do  you  think  any  student  performance  evaluation  could  be  done  in 
the  BT33?  Yes  No  

If  your  answer  is  "Yes,"  when  do  you  think  this  should  take  place? 


As  compared  to  the  live  fire  exercise  situation,  how  well  does 
the  BT33  environment  facilitate  instructor-student  interaction? 


Very 

Well 


As  compared  to  the  live  fire  exercise  situation,  how  well  does 
the  BT33  environment  enable  you  to  identify  the  nature  of  student 
performance  problems  and  misunderstandings? 


8.  How  adequately  can  you  manage  the  BT33  display  presentations? 


Moderately 

Well 


Do  you  have  any  specific  problems  in  managing  the  presentations? 
Yes  No  


Consider  the  fidelity  of  BT33  display  presentations  and  the  vari 
ety  of  situations  you  can  create.  Please  make  a judgment  of 
device  adequacy  and  identify  any  needs  for  improvement,  which, 
if  made,  would  improve  the  device's  training  effectiveness. 


DISPLAY  ITEM 


ADEQUATE 


INADEQUATE 


NEED  FOR 


TARGETS 

(Check  one) 

TERRAIN 

BURSTS 

OTHER: 

Do  you  have  any  other  recommendations  for  improvement  of  either  the 
BT33  itself  or  the  manner  in  which  it  is  used? 


MOS  TIME  IN  THIS  MOS  Months 

GRADE  TIME  IN  SERVICE Months 

SIGNATURE  


The  U SAFAS,  with  the  assistance  of  the  Army  Research  Institute 
(ARI) , is  evaluating  the  utility  of  the  BT33  Fire  Control  Simulator 
for  Forward  Observer  training;  It  is  a Swedish  training  device, 
built  by  Saab-Scania.  It  is  comparable  in  many  ways  to  the  Army's 
own  Observed  Fire  Trainer,  which  is  now  under  development. 


As  an  aid  in  evaluating  the  characteristics  and  utility  of  this 
training  device  as  an  instructional  tool,  please  fill  out  this  cover 
sheet  and  the  attached  questionnaire  as  completely  as  you  can.  Your 
answers  will  be  used  for  research  purposes  only  and  will  not  become 
a part  of  your  record.  When  identifiers  (name  and  Social  Security 
Account  Number)  are  requested  they  are  to  be  used  for  administrative 
and  statistical  control  purposes  only.  Full  confidentiality  of  the 
responses  will  be  maintained  in  the  processing  of  these  data. 


USAFAS  BT33  TRAINING  EFFECTIVENESS  STUDY 
OPERATOR  EVALUATION 


Your  participation  in  this  research  is  strictly  voluntary.  In- 
dividuals are  encouraged  to  provide  complete  and  accurate  information 
in  the  interests  of  this  equipment  evaluation,  but  there  will  be  no 
effect  on  individuals  for  not  providing  all  or  any  part  of  the  infor- 
mation. In  keeping  with  the  Privacy  Act  of  1974,  we  request  that  you 
sign  your  name  at  the  bottom  of  the  cover  sheet  indicating  that  you 
have  not  objected  to  completing  this  form. 


very 

Difficult 


Moderate 


Is  it  easy  or  difficult  to  make  errors  in  the  operation  of  the 
BT33?  I.e. , how  precise  and  careful  do  you  always  have  to  be 
in  order  to  avoid  making  errors? 


Very 
Easy  to 
Make  Errors 


Moderate 


Very 

Difficult  to 
Make  Errors 


If  any  operational  errors  can  be  easily  made,  would  you  please 
describe  what  these  are? 


Is  it  easy  or  difficult  to  avoid  damage  to  the  equipment  when 
operating  the  BT33? 


Many  of  the  questions  will  ask  that  you  rate  the  BT33  on  some 
characteristic  through  use  of  a rating  scale.  Please  consider 
the  scales  to  be  continuous  ones.  Place  am  "x"  on  the  scale 
at  whatever  location  best  describes  your  opinion. 


1.  Compared  to  other  similar  devices  you  have  worked  with,  how  easy 
is  the  BT33  to  operate  overall? 


i 


4.  Are  there  any  other  specific  problems  in  the  layout  or  operation 
of  the  device? 

Yes  

No  

If  your  answer  was  "Yes,"  please  describe  these  problems. 


5.  In  the  selection  of  personnel  to  become  BT33  operators,  how  much 
electronic  equipment  background  experience  do  you  think  they 
should  have? 


Very  Moderate  A Great 

Little  Deal 

Would  you  please  describe  the  desirable  background  for  a BT33 
operator  in  terms  of  MOS,  amount  of  MOS  experience,  grade,  etc.? 


In  the  selection  of  personnel  to  become  BT33  operators,  how  much 
forward  observer  background  experience  do  you  think  they  should 


How  many  days  of  classroom  and  on-the-job  training  do  you  feel  a 
person  would  have  to  have  before  he  would  be  a qualified  BT33 
operator? 

Classroom  days 

OJT  days 


How  frequently  do  you  feel  a person  would  have  to  perform  as  a 
BT33  operator  to  maintain  his  operator  proficiency? 


Once  a 
Month 


Once  a 
Week 


Every 

Day 


Please  describe  the  above  frequency  in  words  (e.g.f  one  week  out 
of  four,  once  a week,  etc.). 


From  an  operator's  standpoint,  do  you  have  any  recommendations  for 
the  improvement  of  the  BT33? 


Table  C-3 

REij^,  REjjj,  and  RE^  Values  for  Experimental 
Condition  Number  Three  (XXS) 


Performance 

Events 

measures 

Mobile  Shoot  One 

Bunker  Shoot 

Mobile  Shoot  Two 

Time  to  Locate 

Target 

1.17 

1.18 

.94 

Total  Mission 

Time 

1.33 

1.47 

.80 

Accuracy  of  Initial 
Target  Location 

1.01 

1.34 

1.05 

Number  of 

Rounds  Fired 

.96 

1.27 

1.04 

Procedural  Errors 
Cut  Score 

1.00 

.67 

.87 

Total  Cuts 

Score 

GO 

• 

1.40 

1.14 

Table  C-4 


REjk,  and  REk  Values  for  Experimental 
Condition  Number  Four  (SSX) 


Performance 

Events 

measures 

Mobile  Shoot  One 

Bunker  Shoot 

Mobile  Shoot  Two 

Time  to  Locate 

Target 

1.03 

.76 

.94 

Total  Mission 

Time 

.99 

.93 

1.11 

Accuracy  of  Initial 
Target  Location 

1.21 

1.14 

1.64 

Number  of 

Rounds  Fired 

1.00 

1.07 

1.10 

Procedural  Errors 

Cut  Score 

1.13 

1.85 

.83 

Total  Cuts 

Score 

.98 

2.24 

1.71 

*v k=4 


RE,  , k=4 
k 


Table  C-5 


RE^jjj,  REj^,  and  RE^  Values  for  Experimental 
Condition  Number  Five  (SXS) 


Performance 

Events 

measures 

Mobile  Shoot  One 

Bunker  Shoot 

Mobile  Shoot  Two 

Time  to  Locate 
Target 

1.67 

1.53 

1.15 

Total  Mission 

Time 

1.58 

1.51 

1.12 

Accuracy  of  Initial 
Target  Location 

.52 

1.78 

1.61 

Number  of 

Rounds  Fired 

.84 

.81 

1.06 

Procedural  Errors 
Cut  Score 

.57 

.84 

1.42 

Total  Cuts 

Score 

.71 

.77 

1.99 

“V  k'5 

BE.,  k-5 
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Table  C-6 

REijjj,  REj]c«  and  RE^  Values  for  Experimental 
Condition  Number  Six  (XSS) 


Performance 

Events 

measures 

Mobile  Shoot  One 

Bunker  Shoot 

Mobile  Shoot  Two 

Time  to  Locate 

Target 

1.03 

1.00 

1.03 

Total  Mission 

Time 

.83 

.66 

1.16 

Accuracy  of  Initial 
Target  Location 

.87 

.54 

1.21 

Number  of 

Rounds  Fired 

.84 

.81 

1.14 

Procedural  Errors 

Cut  Score 

1.19 

3.60 

.94 

Total  Cuts 

Score 

CO 

• 

.66 

1.53 

Table  C-7 

REj.^,  and  RIL  Values  for  Experimental 

Condition  Number  Seven  (SSS) 


Performance 

measures 

Events 

Mobile  Shoot  One 

Bunker  Shoot 

Mobile  Shoot  Two 

Time  to  Locate 

Target 

CM 

• 

1.08 

1.20 

Total  Mission 

Time 

.91 

1.06 

1.29 

Accuracy  of  Initial 
Target  Location 

1.33 

.86 

1.21 

Number  of 

Rounds  Fired 

1.04 

1.43 

• 98 

Procedural  Errors 

Cut  Score 

.87 

1.00 

1.49 

Total  Cuts 

Total  Cuts 
Score 


98 
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1.74 
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1 USA  Air  Oaf  Sch.  Ft  Blisi,  ATTN:  Tacfc  Lib 
1 USA  Air  Def  Bd,  Ft  Bli»,  ATTN:  FILES 
1 USA  Air  Oaf  Bd,  Ft  BKis,  ATTN:  STEBO-PO 
1 USA  Cmd  & General  Stf  Cotlegs,  Ft  Leavenworth.  ATTN:  Lib 
1 USA  Cmd  & General  Stf  College.  Ft  Leavenworth.  ATTN:  ATSW-SE-L 
1 USA  Cmd  & General  Stf  Collage,  Ft  Leavenworth,  ATTN:  Ed  Advisor 
1 USA  Combined  Arms  Cmbt  Dm  Act.  Ft  Leavenworth,  ATTN:  DepCdr 
1 USA  Combined  Arm»  Cmbt  Dm  Act.  Ft  Leavenworth,  ATTN:  CCS 
1 USA  Combined  Arm*  Cmbt  Dm  Act.  Ft  Leavenworth,  ATTN:  ATCASA 
1 USA  Combined  Arms  Cmbt  Dev  Act.  Ft  Leavenworth,  ATTN:  ATCACO-E 
1 USA  Combined  Arms  Cmfct  Dev  Act,  Ft  Leevenworth,  ATTN:  ATCACC-CI 
1 USAECOM.  Night  Vision  Lab.  Ft  Belvoir,  ATTN:  AMSEL-NV-SD 

3 USA  Computer  Sys  Cmd.  Ft  Belvoir.  ATTN:  Tech  Library 
1 USAMERDC,  Ft  Balvoir,  ATTN:  STSFB— OO 

1 USA  Eng  Sch.  Ft  Belvoir.  ATTN:  Library 
1 USA  Topographic  Lab,  Ft  Balvoir,  ATTN:  ETL-TD-S 
1 USA  Topographic  Lab.  Ft  Belvoir,  ATTN:  STINFOCantar 
1 USA  Topographic  Lab.  Ft  Bahrair,  ATTN:  ETL-GSL 
1 USA  Intettigance  Ctr  & Sch,  Ft  Huachuca  ATTN:  CTD-MS 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATS— CTD'JdS 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  AT5I-TE 
1 USA  Intelligence  Ctr  A Sch.  Ft  Huachuca.  ATTN:  ATSI-TEX-GS 
t USA  Intelligence  Ctr  & Sch.  Ft  Huachuca.  ATTN:  ATSI-CTS-OR 
1 USA  Intafligenee  Ctr  & Sch.  Ft  Huachuca.  ATTN:  ATSI— CTO— DT 
1 USA  Intelligence  Ctr  ft  Sch,  Ft  Huachuca.  ATTN:  ATSI-CTD-CS 
1 USA  Intelligence  Ctr  A Sch.  Ft  Huachuca,  ATTN:  DAS/SRD 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  ATSI-TEM 
1 USA  Intelligence  Ctr  A Sch,  Ft  Huachuca,  ATTN;  Library 

1 CDR,  HQ  Ft  Huachuca.  ATTN:  Tech  Ref  Div 

2 CDR.  USA  Electronic  Prvg  Grd,  ATTN:  STEEP-MT-S 
I HQ,  TCATA,  ATTNi  Tacfl  Library 

1 HQ,  TCATA,  ATTN:  AT  CAT-OP-Q,  Ft  Hood 
1 USA  Recruiting  Cmd,  Ft  Sheridan,  ATTNi  USARCPM-P 
1 Senior  Army  Adv..  USAFAOOtVTAC,  Eight  AF  Aux  Fid  No.  • 

1 HQ  USARFAC,  DCSPER,  APO SF  »BSSB.  ATTN*  OPPE-SE 
1 Stlmson  Lib,  Academy  of  Health  Sciences  Ft  Sam  Houston 
1 Marina  Corp.  I net.,  ATTNi  Been  MCI 
1 HQUSMC.  Commandant,  ATTNi  Coda  MTMT 

1 HQUSMC,  Commandant,  ATTNi  Code  M Pi-20-** 

2 USCG  Academy,  New  London,  ATTNi  Miglnlm 
2 USCG  Academy,- New  London,  ATTNi  Library 

1 USCG  Training  Clr,  NY,  ATTNi  CO 
1 USCG  Training  Ctr,  NV,  ATTNi  Educ  Svc  Ofc 
1 USCG.  PiychcM  Rat  Br,  DC,  ATTNi  GPt/ft 
1 HQ  Mld-Ranga  Br,  MC  Oat.  QuanHcct  ATTNi  PAS  Dhr 


4  OASD  IMARA) 

2 HQDA  (DAMI-CSZ) 

1 HQOA  IDAPE-PBR 
1 HQOA  IDAMA-AR) 

1 HQDA  (DAPE-HRE-FO) 

1 HQOA  (SGRO-ID) 

1 HQOA  (OAMI-DOT  O 
1 HQOA  (OAPC-PMZ-A) 

1 HQOA  (DACH-PPZ-A) 

1 HQOA  (DAPE-HRE) 

1 HOOA  (DAPE-MPO-CI 
1 HQOA  (DAP6-OW) 
t HQOA  (DAPE-HPL) 

1 HQOA  (DAPE-CPS) 

1 HQOA  (OAFO-MFA) 

1 HQOA  (DARD-ARS-PI 
1 HQOA  (DAPC-PAS-A) 

1 HQTMDUSA-L  T) 

1 HQOA  (OAMO-RQR) 
t HQOA(DASG) 

1 HQDA  (DAIOPI) 

1 Chief,  Comuit  Diy  (OA-OTSG),  Adeiphi,  MD 
1 Mil  Ast.  Hum  Ret,  OOORAE,  OAO  (E4LS) 

1 HQ  USARAL.  APO  Seattle,  ATTN:  ARAGP-R 

1 HQ  Pint  Army.  ATTN:  AFKA-OI-T1 

2 HQ  Fifth  Army,  Ft  Sam  Houiton 

1 Oir.  Army  Stf  Studies  Ofc,  ATTN:  OAVCSA  fOSP) 
i Ofc  Chief  of  Stf,  Studiei  Ofc 
1 DCSPER,  ATTN:  CPS.GCF 
1 The  Army  Lib,  Pentagon.  ATTN:  RSB  Chief 
1 The  Army  Ub,  Pentagon,  ATTN:  ANRAL 
1 Ofc,  Ant  Sect  of  the  Army  (RSO) 

1 Tech  Support  Ofc.  OJCS 
1 USASA,  Arlington.  ATTN:  IARO-T 

1 USA  Rich  Ofc.  Durham,  ATTN:  Life  Sciences  Dir 

2 USARIEM,  Natick.  ATTN:  SGRO-UE-CA 

t USATTC,  Ft  Clayton,  ATTN:  STETC-MO-A 
1 USAIMA.  Ft  Bragg.  ATTN:  ATSU-CTD-OM 
1 USAIMA.  Ft  Bragg.  ATTN:  Marquet  Lib 
t US  WAC  Ctr  & Sch,  Ft  McClellan.  ATTN:  Ub 
1 US  WAC  Ctr  & Sch.  Ft  McClellan.  ATTN:  Tog  Dir 
1 USA  Quartermaster  Sch.  Ft  Lee.  ATTN:  ATSM-TE 
1 Intelligence  Malarial  Dm  Ofc,  EWl»  Ft  Hotabird 
1 USA  SE  Signal  Sch.  Ft  Gonfon,  ATTN:  ATSO-EA 
1 USA  Chaplain  Ctr  ft  Sch.  Ft  Hamilton.  ATTN:  ATSC-TE-RD 
1 USATSCH,  Ft  Euitii.  ATTN:  Educ  Advisor 

1 USA  War  College.  Carlisle  Barracks.  ATTN:  Lib 

2 WRAIR,  Nauropsychiatry  Div 
1 DLI,  8DA,  Monteray 

I USA  Concept  Anal  Agcy,  Bo  the  id*,  ATTNi  MOCA-MR 
I USA  Concept  Anal  Agcy,  Bothesda,  ATTNi  MOCA-JF 
I USA  Arctic  Tost  Ctr.  APO  Seattle,  ATTNi  STEAC-PL-MI 
1 USA  Arctic  Test  Ctr,  APO  Seattle,  ATTNi  AMSTE-PL-TS 
1 USA  Armament  Cmd,  Restone  Arsenal,  ATTNi  ATSK-TEM 
I USA  Armament  Cmd,  Rock  Island,  ATTNi  AMSAR-TOC 
1 FAA-NAFEC.  Atlantic  City.  ATTNi  Library 
1 FAA-NAFEC,  Atlantic  City,  ATTNi  Hum  Engr  Br 
I FAA  Aeronautical  Ctr,  Oklahoma  City,  ATTNi  AAC-44D 
* USA  Fid  Arty  Sch,  Ft  Sill,  ATTNi  Library 
1 USA  Armor  Sen,  Ft  Knox,  ATTNi  Library 
1 USA  Armor  Sch,  Ft  Knox,  ATTNi  ATSB-OI-E 
I USA  Armor  sch.  Ft  Knox.  ATTNi  ATSB-OT-TT 
t USA  Armor  Sch,  Ft  Knox,  ATTNi  ATSB-CD-AO 


1  US  Marin*  Coma  LiaWon  Ofc,  AMC.  Afcxarabla.  ATTN:  AfctCOB-F 
I UBATRAOOC,  Ft  Monrao,  ATTN:  ATRO— EO 
• UBATRAOOC.  Ft  Monro*.  ATTN:  ATFR-AD 
1 UBATRAOOC,  Ft  Monroa,  ATTN:  ATTS-EA 

1 UBA  FanttOwd.  Ft  UNFIwni.  ATTN:  Lfcoy 

2 UBA  Aatadon  Tact  Bd,  Ft  Rucker.  ATTN:  STEBG-PO 

1 UBA  Agcy  tar  AoMod  Sitaty,  Ft  Rucker,  ATTN:  Library 
1 USA  Agcy  tar  Aaiadon  Sitaty,  Ft  Rucfcor,  ATTN:  EduoAdafcor 
1 USA  Aviadon  Sdi.  Ft  Rucfcor,  ATTN:  PO  Drawer  O 

1 HOUSA  Aviadon  SyaCnd,  St  Loud,  ATTN:  AMSAV-ZDR 

2 USA  IModoh  Syt  TWf  Act,  Edwarda  AFB,  ATTN:  SAVTE-T 
t USA  Air  Def  Sdi,  Ft  BHaa,  ATTN:  ATSATEM 

1 USA  Air  MoMlity  Arch  A Do*  Lab,  MoFtatt  Fid,  ATTN:  SAVDL— AS 
1 UBA  Artetion  Sdi,  Rra  Tog  Mgt,  Ft  Rucfcor.  ATTN:  ATST-T-RTM 
1 USA  Artadon  Sdi,  CO,  Ft  Rucfccr,  ATTN:  ATST- O— A 
1 HQ.  OARCOM.  Alexandria.  ATTN:  AMXCO-TL 
t HQ,  DARCOM.  Alexandria,  ATTN:  COR 
1 US  Military  Academy,  Pfcrt  Point  ATTN:  Sariak Unit 
1 US  Military  Academy,  Watt  Point.  ATTN:  Ofc  ofMiltLdrdip 
1 US  Military  Acodmy.  W«t  Point.  ATTN:  MAOR 
t USA  Standardization  Go.  UK.  FPO  NY,  ATTN:  MASE-GC 
t Ota  of  Nani  Rich,  Arlington,  ATTN:  Code  452 

3 Ofc  of  Navd  Radi.  Arlington.  ATTN:  Code  468 
1 Ota  of  Noral  Rich,  Arlington,  ATTN:  Code  460 
1 Ofc  of  Nani  Rich.  Arlington.  ATTN:  Coda  441 

1 Nani  Aeroape  Mad  Raa  Lab,  Rmacola,  ATTN:  Aeon  Sch  Dfcr 
1 Nani  Aorape  Mad  Raa  Lab,  Fcraacola,  ATTN:  Coda  LSI 
1 Nani  Aeroape  Mad  Rai  Lab.  Pemacok.  ATTN:  Codr  L6 
1 Chiaf  of  NavFen,  ATTN:  ta»OR 
1 NAVAIRSTA.  Norfolk,  ATTN:  Sototy  Ctr 
1 Na»  Oceanographic,  DC,  ATTN:  Code  6261,  Chirk  A Tach 
1 Cantarof  Nani  And,  ATTN:  Doe  Cv 
1 NavAirSyaCont.  ATTN:  AIR-5313C 
1 Na»  BuMad,  ATTN:  713 


1 AFHRL IFT)  William  AF6 
1 AFHRL  rm  Lowry  AFB 

1 AFHRL  IAS)  WPAFB.  OH 

2 AFHRL  IDOJZ)  Brooka  AFB 

1 AFHRL  (OOJN)  Lackland  AFB 
1 HOU8AF  (INYSO) 

1 HQUSAF  IOPXXA) 

1 AFVTG  (RO)  Randolph  AFB 

3 AMRL  (HE)  WPAFB,  OH 

2 AF  Inatof  Tach.  WPAFB.  OH.  ATTN:  ENE/BL 
1 A7C  (XPTDI  Randolph  AFB 

1 USAF  AeroMad  Lib,  Broofca  AFB  ISUL-4).  ATTN:  DOC  SEC 
1 AFOSR  (NL),  Arlington 

1 AF  Log  Cmd,  McCWIan  AFB,  ATTN:  ALC/DFCRS 
1 Air  Foroa  Academy.  CO,  ATTN:  Dapt  of  Bal  Son 


